The “Levels” Problem in Assessing Organizational Climate: Evidence From the Federal Employee Viewpoint Survey

• Author: James R. Thompson,Michael D. Siciliano
• DOI: 10.1177/0091026020917710
• Published date: 01 March 2021
• Date: 01 March 2021
• Subject Matter: Articles

917710PPMXXX10.1177/0091026020917710Public Personnel ManagementThompson and Siciliano
research-article2020
Article
Public Personnel Management
2021, Vol. 50(1) 133 –156
The “Levels” Problem in
© The Author(s) 2020
Article reuse guidelines:
Assessing Organizational
sagepub.com/journals-permissions
https://doi.org/10.1177/0091026020917710
DOI: 10.1177/0091026020917710
journals.sagepub.com/home/ppm
Climate: Evidence From the
Federal Employee Viewpoint
Survey
James R. Thompson1 and Michael D. Siciliano1
Abstract
The Federal Employees Viewpoint Survey (FEVS) has gained prominence as the
primary vehicle through which agencies assess the work-related attitudes of their
employees. Within the discipline, the FEVS results have proven a fertile source of
data on the job-related attitudes of public employees. However, concerns have been
raised with the instrument and with how the FEVS data has been used by scholars.
This article highlights a concern with the use of ambiguous terms which impede
interpretation of the FEVS results. An investigation in partnership with officials from
the regional office of a federal agency confirmed the divergent meanings respondents
assign terms such as “my organization” and brought to the fore the extent to which,
in large organizations, employee attitudes are distinctive to the units to which they
belong at different hierarchical levels. The literature on organizational climate provides
a useful framework within which the phenomenon can be understood and analyzed.
Keywords
organizational climate, Federal Employee Viewpoint Survey, questionnaire design,
multi-level analytical techniques
Scholars have found organizational climate to be an important determinant of both
individual outcomes such as job satisfaction and organizational commitment and
organizational outcomes such as sales performance and customer satisfaction.
Climate, defined as the collective perceptions by members of organizational policies
1The University of Illinois at Chicago, IL, USA
Corresponding Author:
James R. Thompson, Department of Public Administration, University of Illinois at Chicago, 400 S. Peoria
St., Chicago, IL 60607-7064, USA.
Email: jthomp@uic.edu

---

134
Public Personnel Management 50(1)
and practices (Schneider & Snyder, 1975), is commonly measured through the use of
surveys. For example, an organization may administer a survey to its employees ask-
ing their perceptions on subjects such as communication, decision-making practices,
and managerial support. The results of such surveys can provide managers and execu-
tives with important information that can be used to improve work conditions and to
positively impact employee attitudes, motivation, and ultimately performance.
A prominent example of a climate survey in public administration is the Federal
Employee Viewpoint Survey (FEVS) administered annually to a high proportion of
the federal workforce by the U.S. Office of Personnel Management (OPM).1 The
FEVS results have garnered significant interest within the federal community in part
as a consequence of the annual, “Best Places to Work” rankings compiled by the
Partnership for Public Service (PPS, 2019). Three items from the FEVS are used to
compile an index of employee engagement. PPS then ranks agencies in the categories
of “large,” “mid-sized,” and “small.” Agencies at the top of each category such as the
National Aeronautics and Space Administration in the large agency category become
the objects of favorable attention while those at the bottom become the objects of
inquiry on the part of, among others, congressional committees (Wagner, 2017). The
FEVS has also served as an object of attention within the scholarly community; schol-
ars have successfully mined the results as a means of probing relationships between
constructs as diverse as organizational integrity, managerial trustworthiness, and pub-
lic service motivation (Somers, 2017).
Widespread use of the FEVS data has, in turn, resulted in scrutiny of the instrument
and of how the data has been employed for scholarly purposes. In a 2015 article,
Fernandez, Resh, Moldogaziev, and Oberfield provide a broad overview of the FEVS
and its history. They cite a list of 42 articles that had been published in public admin-
istration journals between 2000 and 2013 as evidence of the extent to which scholars
have made use of data from the FEVS or its predecessors. While acknowledging the
value of the FEVS data to scholars, Fernandez et al. are nevertheless critical of the
FEVS on multiple grounds including that the FEVS lacks, “a central focus or guiding
research question” (p. 388). They further criticize some aspects of the design of the
FEVS instrument including, for example, the inclusion of, “double-barreled” (p. 389)
items, that is, items which combine two or more constructs that are appropriately
probed separately. In 2017, Somers offered a separate critique directing attention not
to the design of the FEVS but to how the FEVS data has been used or misused by
scholars. He references multiple constructs that scholars have assembled from FEVS
items such as autonomy, intrinsic motivation, leadership, and job involvement for
which validation procedures have been inadequate.
This article continues the conversation initiated by Fernandez et al. (2015) and
Somers (2017) about the FEVS and its use. Similar to Fernandez et al., we highlight
questions with regard to the construction of some FEVS items. Similar to Somers, we
highlight the need for scholars to use the FEVS data consistent with its intended pur-
pose which is to serve as a means of assessing organizational climate.
The relevance of that purpose became apparent in the course of an investigation
undertaken by the authors in partnership with officials from the regional office of a

---

Thompson and Siciliano
135
federal agency. One purpose of the partnership was to investigate issues of work moti-
vation in the office. A second purpose had to do with the difficulties these officials had
encountered in interpreting the results of some of the items in the FEVS. A number of
the FEVS items, like other climate surveys, use terms such as “your [or my] organiza-
tion” where the precise referent is unclear. For example, one of the items in the FEVS
reads, “Considering everything, how satisfied are you with your organization?” The
problem, as cited by these officials, was that, “your organization” could refer to any
one of five different organizational levels, section (lowest level), branch, division,
regional office, or the agency as a whole. Unable to discern from the FEVS results how
employees interpret the term, these officials have been at a loss to know, when the
FEVS results signaled a problem that needed remediation, at which level to intervene.
Table 1 lists the eleven FEVS items for which the precise referent is unclear.
Table 1. Ambiguous Terms in the Federal Employee Viewpoint Survey.
Term
Examples
“my organization”
“I recommend my organization as a good place to work.” (p. 40)

“Considering everything, how satisfied are you with your organization?”
(p. 71)
“leaders”
“In my organization, leaders generate high levels of motivation and
commitment in the workforce.” (p. 53)

“My organization’s leaders maintain high standards of honesty and
integrity.” (p. 54)
“managers”
“Managers communicate the goals and priorities of the organization.”
(p. 56)

“Managers review and evaluate the organization’s progress toward
meeting its goals and objectives.” (p. 57)

“Managers promote communication among different work units
(for example, about projects, goals, needed resources.” (p. 58)

“Managers support collaboration across work units to accomplish work
objectives.” (p. 59)
“senior leaders”
“I have a high level of respect for my organization’s senior leaders.”
(p. 61)

“How satisfied are you with the policies and practices of your senior
leaders?” (p. 66)
“management”
“How satisfied are you with the information you receive from
management on what’s going on in your organization?” (p. 64)
Our probe revealed that employees within the regional office assign highly diver-
gent meanings to terms such as “your organization.” Although the immediate issue is
one of survey design and cognitive processing, the broader problem has to do with the
issue of how climate is assessed in large organizations with multiple hierarchical lev-
els. Is climate relevant only to the organization as a whole or can distinctive climates
exist within subsidiary units? The answer that emerges from the climate literature is

---

136
Public Personnel Management 50(1)
that a distinctive climate exists within any one organizational unit only to the extent
that it can be demonstrated (a) that there is some minimal degree of consensus as to the
nature of that climate among employees within that unit and (b) there are differences
in climate perceptions across units at the same level.
In the section that follows, we present an overview of the climate construct which
has received limited attention within the public administration literature. We then
provide a review of how the FEVS has evolved and of how OPM is seeking to bring
the FEVS into conformity with more conventional climate assessment instruments.
Organizational Climate
Scholars have found organizational climate to be consequential for both individual and
organizational outcomes. Kuenzi and Schminke (2009) cite 89 empirical works pub-
lished between 1980 and 2008...