State Agencies’ Use of Administrative Data for Improved Practice: Needs, Challenges, and Opportunities

• Date: 01 March 2018
• Published date: 01 March 2018
• DOI: http://doi.org/10.1111/puar.12883

240 Public Administration Review • March | April 2018

Public Administration Review,

Vol. 78, Iss. 2, pp. 240–250. © 2017 by

The American Society for Public Administration.

DOI: 10.1111/puar.12883.

State Agencies’ Use of Administrative

Data for Improved Practice:

Needs, Challenges, and Opportunities

Colleen Schlecht is researcher at Chapin Hall

at the University of Chicago. Her work includes

qualitative research and quantitative analysis of

programs and supports for vulnerable youth, from

early childhood to emerging adults, and their

families. She holds a master’s degree in public policy

from the University of Chicago and a bachelor’s

degree in public policy studies from Duke University.

E-mail: cschlecht@chapinhall.org

Emily R. Wiegand is researcher at Chapin

Hall at the University of Chicago, specializing in

data manipulation and management, database

development, quantitative analysis, and the use of

administrative data to guide strategic planning and

decision-making in the nonprofit and public sectors.

She has a master’s degree in public policy and a

bachelor’s degree in history, both from University

of Chicago.

E-mail: ewiegand@chapinhall.org

Scott W. Allard is professor in the Evans School

of Public Policy and Governance at the University of

Washington. He is a nonresident senior fellow in the

Brookings Institution’s Metropolitan Policy Program

and codirector of the Family Self-Sufficiency Data

Center at the University of Chicago.

E-mail: sallard@uw.edu

Abstract : Growing interest in the use of administrative data to answer questions around program implementation

and effectiveness has led to greater discussion of how government agencies can develop the necessary internal data

infrastructure, analytic capacity, and office culture. However, there is a need for more systematic research into how

states find different pathways and strategies to build administrative data capacity. Drawing on interviews with almost

100 human service agency staff and their data partners, the authors examine the realities of administrative data

use. They summarize the experiences of data users in order to address two main challenges : limited analytic capacity

and challenges to linking or sharing data resources. The article concludes by examining a range of approaches that

government agencies take to improve data quality and capacity to analyze that data .

Evidence for Practice

• Improving data quality tends to involve intentional but incremental efforts to enhance the accuracy of data

entry, reduce the prevalence of missing data, and create detailed data documentation.

• High-capacity administrative data users allocate staff time to reflect on questions or metrics that extend

beyond mandatory reports and will have immediate benefits to the agency and its clients.

• Agencies making consistent commitments to analytic activity outside of mandatory reporting do so by

prioritizing and protecting staff time for such activities and developing data infrastructures that make data

flexibly available for analytic work.

• Integrating analytics into program planning and design often requires intentional partnerships with external

researchers, agencies in other states, charitable philanthropy, and federal agencies.

• States seeking to become field-leading learners build and share comprehensive data resources, participate in

creative discussions about data analytics and program design, and support the learning of data users.

A lthough researchers and government

agencies have been using administrative

data to understand program dynamics for

several decades, there is growing interest in the use

of administrative data to answer questions around

program implementation and effectiveness across

a range of social policies. A number of factors have

increased the role that administrative data play in

policy-driven research today: more accessible and

powerful data analysis tools, a growing emphasis

on evidence-based policy, the use of data-driven

decision-making tools in program management,

and data science or “big data” initiatives intended to

improve data resources and capacity (see Isett, Head,

and VanLandingham 2016 ; Lavertu 2016 ; Mergel,

Rethemeyer, and Isett 2016 ). Momentum behind

greater administrative data use also builds on recent

work to better understand social policy dynamics

(Einav and Levin 2013 ; Johnson, Massey, and O’Hara

2014 ), the consequences of welfare reform (Cancian

et al. 2002 ; Lee, Mackey-Bilaver, and Goerge 2003 ;

Ribar, Edelhoch, and Liu 2016 ), the impact of

subsidized child care (Weber, Grobe, and Davis

2014 ), and the relationship between neighborhood

characteristics and employment (Chetty, Friedman,

and Saez 2013 ).

Nevertheless, common limitations of administrative

data can hinder their contribution to evidence-

based policy decision-making processes (Hotz

et al. 1998 ). These limitations generally fall into

two main groups: limits on valid inference and

data use obstacles. The validity of inference from

administrative data can be compromised for a variety

of reasons (Greer and Bullock 2017 ; Lavertu 2016 ;

Mergel, Rethemeyer, and Isett 2016 ). Data systems

not initially designed for performance measurement

may not yield administrative data containing accurate

information about key performance or outcome goals.

Information located in administrative data files may

not be checked regularly for accuracy or consistency.

Sample restrictions also exist because administrative

Scott W. Allard

University of Washington

Emily R. Wiegand

Colleen Schlecht

Chapin Hall at the University of Chicago

A. Rupa Datta

NORC at the University of Chicago

Robert M. Goerge

Chapin Hall at the University of Chicago

Elizabeth Weigensberg

Mathematica Policy Research

A. Rupa Datta is vice president and senior fellow

at NORC at the University of Chicago, where she leads

major studies that combine surveys with administrative

and commercial data sources to support policy analysis

in human services such as employment supports, early

care and education, and poverty alleviation. She liaises

with state administrators as part of the Family Self-

Sufficiency Data Center at the University of Chicago.

E-mail: datta-rupa@norc.uchicago.edu

Robert M. Goerge is senior research fellow at

Chapin Hall at the University of Chicago. He is also a

senior fellow at the Harris School of Public Policy and

the University of Chicago Computation Institute and

codirector of the Family Self-Sufficiency Data Center

at the University of Chicago.

E-mail: rgoerge@chapinhall.org

Elizabeth Weigensberg is senior researcher

at Mathematica Policy Research. She has more than

15years of experience conducting evaluations and

research in the fields of workforce development and child

welfare for federal, state, and local government agencies

and foundations. Her expertise includes linking and

analyzing complex administrative data from state and

local public agencies and providing technical assistance

to develop performance measures and to facilitate the

use of data to inform policy and practice.

E-mail: eweigensberg@mathematica-mpr.com