We Need to Compare, but How? Measurement Equivalence in Comparative Public Administration

Date01 January 2015
Published date01 January 2015
Sebastian Jilke is a postdoctoral
researcher at Erasmus University Rotterdam,
the Netherlands. His research interests
include citizen attitudes and behaviors with
respect to public services, administrative
reforms, and research methodology. He
has published articles on these topics
in European Journal of Political
Research, Public Administration, and
Public Management Review. He is also
coeditor of a forthcoming symposium in
PAR on the use of experiments in public
administration research.
E-mail: jilke@fsw.eur.nl
Bart Meuleman is assistant professor
in the Centre for Sociological Research,
University of Leuven, Belgium, where he
teaches research methodology. His main
research interests involve cross-cultural
comparisons of attitude and value patterns,
such as welfare attitudes, ethnocentrism,
religiosity, and basic human values.
His research has appeared in Annual
Review of Sociology, Public Opinion
Quarterly, Journal of Cross-Cultural
Psychology, and Journal of European
Social Policy.
E-mail: bart.meuleman@soc.kuleuven.be
Steven Van de Walle is professor of
comparative public administration and man-
agement at Erasmus University Rotterdam,
the Netherlands. His research focuses on
interactions between citizens and public
services, trust, and public sector reform. He
was coordinator of the large-scale COCOPS
project (2010–14), a European research
project on public sector reform in which 11
European universities collaborated.
E-mail: vandewalle@fsw.eur.nl
36 Public Administration Review • January | February 2015
Public Administration Review,
Vol. 75, Iss. 1, pp. 36–48. © 2014 by
The American Society for Public Administration.
DOI: 10.1111/puar.12318.
that are relevant for public administration research,
such as the International Social Survey Programme,
the Eurobarometer, the COCOPS (Coordinating for
Cohesion in the Public Sector of the Future) survey
of public managers, or the Consolidated Omnibus
Budget Reconciliation Act (COBRA) survey of
government agency executives, among many others.
Making use of such cross-national survey data gives
us the opportunity to test the geographic range of
social theories by assessing them in many dif‌f erent
contexts. Moreover, having survey data from numer-
ous countries enables us to investigate various micro-
macro relations by utilizing data from the individual
and the country level. Such cross-level interactions
permit us to look more closely at interesting relation-
ships between context and individuals, allowing us to
explicitly test contextual theories.
However, when respondents in dif‌f erent countries
regard measurement constructs in dif‌f erent manners
or exhibit culturally inf‌l uenced response patterns, we
typically obtain biased survey measures (Poortinga
1989; Van de Vijver and Leung 1997). Practically
speaking, the response of a person in country A—say,
to the item on satisfaction we used as an example—
may have the same scale position as the response of
another person in country B, but it could mean some-
thing entirely dif‌f erent if the ways in which respond-
ents interpret or respond to it dif‌f er substantially. By
simply looking at mean levels of survey responses,
Abstract: In addition to public administrations and public managers, there is increasing interest in studying citizens
interactions with and views toward government from a comparative perspective in order to put theories to the test
using cross-national surveys. However, this will only succeed if we adequately deal with the diverse ways in which
respondents in dif‌f erent countries and regions perceive and respond to survey measures.  is article examines the con-
cept of cross-national measurement equivalence in public administration research and explores methods for establishing
equivalence. Two methodologies are examined that test and correct for measurement nonequivalence: multiple-group
conf‌i rmatory factor analysis and multilevel mixture item response theory.  ese techniques are used to test and establish
the cross-national measurement equivalence of two popular measurement constructs: citizen satisfaction with public
services and trust in public institutions. Results show that appropriately dealing with nonequivalence accounts for dif-
ferent forms of biases that otherwise would be undetected.  e article contributes to the methodological advancement
in studying public administration beyond domestic borders.
We Need to Compare, but How? Measurement Equivalence
in Comparative Public Administration
Michael McGuire, Editor
Sebastian Jilke
Erasmus University Rotterdam, the Netherlands
Bart Meuleman
University of Leuven, Belgium
Steven Van de Walle
Erasmus University Rotterdam, the Netherlands
Consider the following survey item: “Overall,
how satisf‌i ed are you with your electricity
supplier? Please give me score from 0 to 10,
where 0 means that you are not satisf‌i ed at all, and 10
means that you are fully satisf‌i ed.”  is is one out of
a battery of items that taps citizen satisfaction with
public services across a wide range of countries.  e
underlying assumption of asking respondents in dif-
ferent national populations the same questions is that
their answers are supposed to be comparable. In other
words, it is assumed that perceptions of what satisfac-
tion means and the way in which people use assigned
scales are equivalent across countries, allowing for
meaningful comparisons. But is the general notion of
what a satisfactory public service is really equivalent
across countries, regions, (groups of) individuals, or
even over time? And are patterns of response styles
the same across dif‌f erent cultures? In this article, we
introduce two major techniques for detecting and
correcting for nonequivalence in the f‌i eld of public
administration, and we show how these methods can
be implemented in applied research.
Comparisons across countries of public administra-
tions, public managers, and interactions and attitudes
of citizens toward government are gaining ground in
public administration research (e.g., Jilke 2014; Kim
et al. 2013; Pollitt and Bouckaert 2011; Van Ryzin
2011).  is is accompanied by an increase in the avail-
ability of cross-national surveys that contain questions

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT