Strategies for Improving Measurement Models for Secondary Data in Public Administration Research: Illustrations from the Federal Employee Viewpoint Survey

• DOI: http://doi.org/10.1111/puar.12805
• Author: Mark John Somers
• Published date: 01 March 2018
• Date: 01 March 2018

228 Public Administration Review • March | April 2018

Public Administration Review,

Vol. 78, Iss. 2, pp. 228–239. © 2017 by

The American Society for Public Administration.

DOI: 10.1111/puar.12805.

Strategies for Improving Measurement Models for

Secondary Data in Public Administration Research:

Illustrations from the Federal Employee Viewpoint Survey

Mark John Somers is professor

of management and organizational

behavior in the Martin Tuchman School

of Management, New Jersey Institute of

Technology. His research interests include

research methods, psychometrics, work

attitudes, employee socialization, and work

outcomes.

E-mail: mark.somers@njit.edu

Abstract : This article builds on Fernandez et al.’s 2015 review of Federal Employee Viewpoint Survey ( FEVS )

research by focusing on two unexplored areas: measurement models and measurement quality. Employing the notion

of an auxiliary measurement theory as an organizing framework, the author assesses the use of FEVS survey items to

operationalize theoretical constructs and procedures in order to establish their psychometric quality. Results indicate

that there is considerable variability in how FEVS items have been used to measure theoretical constructs, which is

expressed as high levels of overlap across FEVS -derived scales. Inconsistency in the assessment of measurement quality

is evident as well, with a bias toward convergent validity. Three cautionary tales are presented to demonstrate the

fragility of FEVS data when used with a compromised auxiliary measurement theory. The article concludes with

recommendations for future FEVS studies.

P ublic administration scholars have made good

use of employee surveys developed by the U.S.

Office of Personnel Management to study

important topics, including agency performance,

employee empowerment, leadership, diversity,

employee turnover, job satisfaction, and fairness. A

commonly used data source in public administration

research is the Federal Employee Viewpoint Survey

(FEVS). Indeed, a fairly

extensive and growing body

of scholarly research based on

FEVS data is evident in the

public administration literature.

In a recent review, Fernandez

et al. ( 2015 ) note that while

the FEVS has been used widely

in public administration research, an evaluation of

this survey and the body of literature it has generated

had not been conducted. Fernandez and colleagues

address this gap by conducting an evaluation of

FEVS research that includes codification of primary

topic areas studied and suggestions about how the

FEVS might be improved to facilitate academic

research. Although several themes run through this

important article, one that recurs is the necessity of

establishing the validity of measures of theoretical

constructs operationalized using FEVS survey

items. The question of validity is important because

FEVS survey items were not designed to measure

the theoretical constructs that are characteristic

of academic research. Rather, because the FEVS

items were crafted as single-item measures to assess

employee perceptions (see Kraut 1996 ), measures

of theory-derived constructs must be constructed a

posteriori.

In discussing the weaknesses and limitations of FEVS

data, Fernandez et al. note that because the FEVS was

designed to improve organizational effectiveness by

tracking employee perceptions, “important concepts

[are] not being measured

properly or not measured at

all” (2015, 388). They go on

to note that validity is rarely

discussed in research using

FEVS data. However, these

concerns are not addressed

in detail because the broad

scope of their review precludes

a more thorough analysis of measurement issues in

FEVS research. Thus, the implications of the limited

attention to measurement quality in FEVS research

remain an important but mostly unexplored area.

The present article addresses this gap in the literature.

Using the notion of an auxiliary measurement theory

(Blalock 1982 ), the development and validation of

measurement models in FEVS research were assessed.

In so doing, an analysis of the implications of slippage

between conceptual and operational variables for

findings from FEVS studies was conducted both as a

means of improving FEVS research and developing

higher-quality measurement models in public

administration research in general. Emphasis was

placed on how constructs were defined and the degree

Mark John Somers

New Jersey Institute of Technology

 e implications of the limited

attention to measurement qual-

ity in FEVS research remain an

important but mostly unex-

plored area.