Assessing Survey-Based Measurement of Personnel Red Tape With Anchoring Vignettes
| DOI | 10.1177/0734371X14531988 |
| Author | Justin Marlowe,Sanjay K. Pandey |
| Published date | 01 September 2015 |
| Date | 01 September 2015 |
| Subject Matter | Articles |
research-article2014
Article
Review of Public Personnel Administration
2015, Vol. 35(3) 215 –237
Assessing Survey-Based
© The Author(s) 2014
Reprints and permissions:
Measurement of Personnel
sagepub.com/journalsPermissions.nav
DOI: 10.1177/0734371X14531988
rop.sagepub.com
Red Tape With Anchoring
Vignettes
Sanjay K. Pandey1 and Justin Marlowe2
Abstract
Despite an upsurge of red tape research, a central issue remains unresolved. The
most widely used red tape measures draw on key informant reports about red tape.
The starkest objection to such measures is that key informant reports are mere
perceptions—perceptions that are subject to distortion. We assess the validity of
key informant perception-based measures of personnel red tape by using “anchoring
vignettes.” Findings suggest that anchoring vignettes can be used to improve the
accuracy of survey measures of red tape. Implications of findings for red tape
scholarship and survey measurement in public management are discussed.
Keywords
Methodological issues, employee attitudes, red tape, anchoring vignettes, perceptual
measures, HR benchmarking/best practices, workplace environment/culture, local
government HRM
Introduction
A series of articles in the early 1990s by Bozeman and colleagues reinvigorated
research on bureaucratic red tape as a salient public management theme (Bozeman,
1993; Bozeman, Reed, & Scott, 1992; Bozeman & Scott, 1996). Bozeman urged pub-
lic management scholars to move beyond merely offering apologies for red tape in the
public sector and to devote their energies to improving empirical research on red tape.
He offered a definition of red tape that has been widely used since: “rules, regulations,
and procedures that remain in force and entail a compliance burden for the
1Rutgers University–Newark, NJ, USA
2University of Washington, WA, USA
Corresponding Author:
Sanjay K. Pandey, Rutgers University–Newark, 111 Washington Street, Newark, NJ 07102, USA.
Email: skpandey@newark.rutgers.edu
216
Review of Public Personnel Administration 35(3)
organization but have no efficacy for the rules’ functional object” (Bozeman, 1993, p.
283). Although modifications have been offered to this definition, the idea of admin-
istrative burden without commensurate benefits is at the center of how red tape is
conceptualized (Brewer & Walker, 2010; Pandey & Scott, 2002).
As research on bureaucratic red tape, in the wake of Bozeman’s foundational con-
tributions, has burgeoned, so have criticisms (Kelman, 2008; Luton, 2007) and caveats
about perceptual measures used in this newly resurgent red tape scholarship (Walker
& Brewer, 2008, 2009). The critiques take issue with the kind of judgments respon-
dents are asked to make to answer survey questions on bureaucratic red tape and sug-
gest that self-reported measures may be unreliable (see Yang & Pandey 2009: 337-339
for a discussion of perceptual vs. objective measures). Caveats by scholars studying
red tape, unsure of where the critical thrust and parry may lead, dutifully note the use
of perceptual measures as a limitation (e.g., Walker & Brewer, 2009).
Addressing this question about reliability of survey questions is important because
there is a growing literature on managerial perceptions of red tape and its relevance for
managerial and organizational action (Baldwin, 1990; Brewer & Walker, 2010; Feeney
& Bozeman, 2009; Moynihan, Pandey, & Wright, 2012; Pandey & Moynihan, 2006;
Pandey & Bretschneider, 1997; Rainey, Pandey, & Bozeman, 1995; Walker & Brewer,
2008, 2009; Welch & Pandey, 2007). But for meaningful cumulation and integration of
findings based on survey research, let alone action based on such knowledge, concerns
about reliability of survey questions used to measure red tape deserve further scrutiny.
We use the anchoring vignette technique developed by King, Murray, Salomon, and
Tandon (2004) to assess a questionnaire item on promotion that is typically used either
by itself or as part of a five-item measure of personnel red tape (see Pandey & Scott,
2002). We expect this perceptual measure of red tape in the promotion process to be
comprised of both true score and random error. By developing vignettes for red tape in
the promotion process—that are consistent with the way red tape concept is defined in
the public management literature (Bozeman, 1993; Pandey & Scott, 2002)—and using
them to apply a correction for differential item functioning (DIF), we expect to increase
the precision and reliability of the item measuring red tape in the promotion process.
Therefore, the goal of this manuscript is to report on the development and testing of
vignettes, applying a correction based on the vignettes, and drawing the implications
of this study for use of survey-based measures of red tape.
Rest of the article is structured as follows. First, we provide a brief history of red
tape research and follow it with an overview of measurement of personnel red tape.
This is followed with contrasting theoretical perspective on how respondents may
respond to personnel red tape survey questions. Next, we describe the data collection
process and present the results of our analyses. We conclude the article by discussing
key findings, implications, and suggestions for future research.
The Red Tape Concept and Survey Measures of
Personnel Red Tape
The reservations about relying on perceptions to assess red tape are by no means of
recent vintage. Noted public administration scholars over time have made the point
Pandey and Marlowe
217
that assessments of bureaucratic red tape pit one person’s opinion against another’s (H.
Kaufman, 1977; Waldo, 1946).1 Gouldner (1952), the renowned sociologist, goes far-
ther than others in this matter and dismisses the possibility of coming up with an inter-
subjectively valid scheme for measuring red tape. According to Gouldner, beneath the
patina of concern with efficiency and burdensome administrative procedures, there is
a fierce contest among competing social values.
How does the recently resurgent scholarship on bureaucratic red tape that uses per-
ception-based measures of red tape address the issues raised by earlier scholars such
as Gouldner, Kaufman, and Waldo among others? The short answer is that it does not
do so directly. The closest head-on engagement of this issue—to the best of our knowl-
edge—is by Pandey (1995) who argues that implicit in classical critiques is a “straw
dog” that “by pointing the divergence between clients’ and managers’ conception of
red tape” (pp. 5-8) concludes that it is not possible to get at inter-subjectively valid
reality of red tape. Pandey recommends instead that red tape be examined as socially
constructed reality that is dependent on the individual’s social role. Bozeman and
Scott (1996), in fact, propose that red tape can be defined only from the perspective of
a “given set of stakeholders” (p. 9). In other words, we should study red tape that man-
agers face and also red tape that clients (or other stakeholders) face rather than con-
tinue the quixotic quest to label something red tape if and only if managers and other
stakeholders can come to full agreement.2
Indeed, recent public management scholarship has primarily focused on managerial
perceptions of red tape, more so in the domain of public personnel systems (Bozeman
& Feeney, 2011; Pandey & Scott, 2002). But can we, therefore, assume that this lim-
ited set of internal organizational actors (as contrasted with comparisons of internal
organizational actors’ perceptions with those of extra-organizational actors) will report
consistent perceptions in answering survey questions? Implicit in classical pre-Boze-
man scholarship on red tape—either using personality concepts such as the “bureau-
cratic personality” (Merton, 1952) or recognizing the role of values in red tape
assessment (Gouldner, 1952)—is the idea that individuals have differing responses to
similar underlying circumstances and therefore are likely to have divergent percep-
tions. More specifically, with respect to perceptions of personnel red tape, Pandey and
Welch (2005) note,
First, managers recognize that personnel systems are designed to serve more than the
values of efficiency and speed. Particularly, public managers are likely to appreciate the
complexity of personnel systems designed to serve a range of valued social goals . . . (p.
547)
Distilled to its essence, the fundamental reason for believing that answers to survey
questions on personnel red tape may be unreliable is that individuals may go beyond
their individual experience of personnel red tape and instead take cognitive shortcuts
to answer the survey questions. This may happen for a number of reasons such as not
having the requisite knowledge or the necessary motivation to answer the questions.
Although bureaucratic red tape can manifest itself in different organizational sys-
tems (Bozeman, 2000; Coursey & Pandey, 2007; Pandey & Scott, 2002), personnel
218
Review of Public Personnel Administration 35(3)
red tape has attracted the most research attention (Baldwin, 1990; Bozeman et al.,
1992; Bretschneider, 1990; Feeney & Rainey, 2010; Pandey & Kingsley, 2000; Pandey
& Moynihan, 2006; Walker & Brewer, 2009). Survey questions typically focus on
common tasks in personnel administration such as recruitment, discipline, termina-
tion, administration of pay raises,...
To continue reading
Request your trial