Paying Attention to the Trees in the Forest, or a Call to Examine Agency-Specific Stories

• Author: Ellen V. Rubin,Keith P. Baker
• Published date: 01 December 2019
• Date: 01 December 2019
• DOI: http://doi.org/10.1177/0734371X17753865
• Subject Matter: Articles

https://doi.org/10.1177/0734371X17753865

Review of Public Personnel Administration

2019, Vol. 39(4) 523 –543

© The Author(s) 2018

Article reuse guidelines:

sagepub.com/journals-permissions

DOI: 10.1177/0734371X17753865

journals.sagepub.com/home/rop

Article

Paying Attention to the

Trees in the Forest, or a

Call to Examine Agency-

Specific Stories

Ellen V. Rubin1 and Keith P. Baker2

Abstract

Public administration scholarship needs to strike a better balance between large

sample studies and in-depth case studies. The availability of large data sets has led

us to engage in empirical research that is broad in scope but is frequently devoid of

rich context. In-depth case studies can help to explain why we observe particular

relationships and can help us to clarify gaps and inconsistencies in theory. Our

argument for more case studies aims to encourage researchers to bridge insights

from qualitative and quantitative research through triangulation. We describe the

value of case study research, and qualitative and quantitative design options. We then

propose opportunities for case study research in public personnel scholarship on

patronage pressures, performance management, and diversity management.

Keywords

case studies, critical and deviant cases, quasi-experimental design, patronage, diversity

management, performance management

Introduction

The availability of large data sets like the Federal Employee Viewpoint Surveys and

other sources of agency management and performance data creates incentives for

public personnel and public management researchers to design studies that prioritize

1University at Albany, NY, USA

2Oregon State University, Corvallis, USA

Corresponding Author:

Ellen V. Rubin, Associate Professor, Department of Public Administration and Policy, University at

Albany, State University of New York, 135 Western Ave., Milne Hall 101, Albany, NY 12222, USA.

Email: erubin@albany.edu

753865ROPXXX10.1177/0734371X17753865Review of Public Personnel AdministrationRubin and Baker

research-article2018

524 Review of Public Personnel Administration 39(4)

breadth above substantive depth. The reliance on large sample studies has produced a

situation in which researchers lack knowledge of the nuances of the operating or polit-

ical environment of the agencies studied. Failure to understand context can result in

“inaccurate inferences” and “inaccurate interpretations” (Durant & Rosenbloom,

2017, p. 725). Furthermore, this lack of contextual knowledge increases the likelihood

that we might claim a successful innovation in one context is the solution to a similar

problem in a different context with different legal, political, and/or administrative tra-

ditions (Pollitt, 2006). As we become more data-rich, we are simultaneously becoming

context-poor.

From an optimistic view, the availability of large sample data sets across agencies

has supported considerable scholarship into bureaucratic behavior, the actions and

motivations of employees, and the impact of various policy agendas (see, for example,

Cohen, Blake, & Goodman, 2016; Meier & Hicklin, 2008; Nicholson-Crotty,

Nicholson-Crotty, & Fernandez, 2017). More data, and more data over time, allow the

field to inch closer to the standards of experimental research from the natural sciences

as a way to signal our rigor and claim legitimacy as a field of scientific study.

However, overreliance on cross-agency studies with inadequate understanding of

the operational realities may lead to significant problems with internal and construct

validity. Take, for example, the public personnel and public management scholarship

on diversity management. Four different reviews of diversity management research

have all called for more detailed study within organizations to assess the degree to

which diversity management policies impact individual, group, and organizational

outcomes (Ivancevich & Gilbert, 2000; Pitts & Wise, 2010; Sabharwal, Levine, &

D’Agostino, 2016; Wise & Tschirhart, 2000). Despite these calls, published research

claiming to be focused on diversity management drawing from U.S. federal agencies

provides very little detail on the deliberate policies implemented by public agencies.

Furthermore, an unclear or imprecise definition of diversity management could lead

scholars to select measures that actually represent organizational culture or manage-

ment support for diversity, rather than actually assessing the effectiveness of imple-

mented policy. Questions and measures that are not thoughtful about the operational

reality in agencies may lead to misinterpreting results and drawing the wrong lessons

for theory (see Riccucci, 2010, for a further discussion).

The central purpose of this article is to describe how studies of individual agencies

can be designed in a manner that are empirically defensible and inform theory devel-

opment. Agency-specific case studies, whether using quantitative, qualitative, or

mixed-methods designs, are particularly useful tools for exploring research questions

in public personnel and public management scholarship. The “Balkan-ization” of gov-

ernment personnel systems resulting from piecemeal reform efforts in agencies with

privileged political status or in agencies under the spotlight of scandal creates impor-

tant opportunities to understand how the unique rules and situations matter—but only

if we know about the details of those reforms and/or scandals. For example, case-

based research might consider the impact of patronage pressures in hiring practices at

U.S. Department of Justice (DOJ) under the Bush administration (DOJ Office of the

Inspector General, 2008) or whether incentive systems at the U.S. Department of