We Need to Compare, but How? Measurement Equivalence in Comparative Public Administration

• Author: Bart Meuleman,Sebastian Jilke,Steven Van de Walle
• Date: 01 January 2015
• Published date: 01 January 2015
• DOI: http://doi.org/10.1111/puar.12318

Sebastian Jilke is a postdoctoral

researcher at Erasmus University Rotterdam,

the Netherlands. His research interests

include citizen attitudes and behaviors with

respect to public services, administrative

reforms, and research methodology. He

has published articles on these topics

in European Journal of Political

Research, Public Administration, and

Public Management Review. He is also

coeditor of a forthcoming symposium in

PAR on the use of experiments in public

administration research.

E-mail: jilke@fsw.eur.nl

Bart Meuleman is assistant professor

in the Centre for Sociological Research,

University of Leuven, Belgium, where he

teaches research methodology. His main

research interests involve cross-cultural

comparisons of attitude and value patterns,

such as welfare attitudes, ethnocentrism,

religiosity, and basic human values.

His research has appeared in Annual

Review of Sociology, Public Opinion

Quarterly, Journal of Cross-Cultural

Psychology, and Journal of European

Social Policy.

E-mail: bart.meuleman@soc.kuleuven.be

Steven Van de Walle is professor of

comparative public administration and man-

agement at Erasmus University Rotterdam,

the Netherlands. His research focuses on

interactions between citizens and public

services, trust, and public sector reform. He

was coordinator of the large-scale COCOPS

project (2010–14), a European research

project on public sector reform in which 11

European universities collaborated.

E-mail: vandewalle@fsw.eur.nl

Research

Synthesis

36 Public Administration Review • January | February 2015

Public Administration Review,

Vol. 75, Iss. 1, pp. 36–48. © 2014 by

The American Society for Public Administration.

DOI: 10.1111/puar.12318.

that are relevant for public administration research,

such as the International Social Survey Programme,

the Eurobarometer, the COCOPS (Coordinating for

Cohesion in the Public Sector of the Future) survey

of public managers, or the Consolidated Omnibus

Budget Reconciliation Act (COBRA) survey of

government agency executives, among many others.

Making use of such cross-national survey data gives

us the opportunity to test the geographic range of

social theories by assessing them in many dif‌f erent

contexts. Moreover, having survey data from numer-

ous countries enables us to investigate various micro-

macro relations by utilizing data from the individual

and the country level. Such cross-level interactions

permit us to look more closely at interesting relation-

ships between context and individuals, allowing us to

explicitly test contextual theories.

However, when respondents in dif‌f erent countries

regard measurement constructs in dif‌f erent manners

or exhibit culturally inf‌l uenced response patterns, we

typically obtain biased survey measures (Poortinga

1989; Van de Vijver and Leung 1997). Practically

speaking, the response of a person in country A—say,

to the item on satisfaction we used as an example—

may have the same scale position as the response of

another person in country B, but it could mean some-

thing entirely dif‌f erent if the ways in which respond-

ents interpret or respond to it dif‌f er substantially. By

simply looking at mean levels of survey responses,

Abstract: In addition to public administrations and public managers, there is increasing interest in studying citizens’

interactions with and views toward government from a comparative perspective in order to put theories to the test

using cross-national surveys. However, this will only succeed if we adequately deal with the diverse ways in which

respondents in dif‌f erent countries and regions perceive and respond to survey measures.  is article examines the con-

cept of cross-national measurement equivalence in public administration research and explores methods for establishing

equivalence. Two methodologies are examined that test and correct for measurement nonequivalence: multiple-group

conf‌i rmatory factor analysis and multilevel mixture item response theory.  ese techniques are used to test and establish

the cross-national measurement equivalence of two popular measurement constructs: citizen satisfaction with public

services and trust in public institutions. Results show that appropriately dealing with nonequivalence accounts for dif-

ferent forms of biases that otherwise would be undetected.  e article contributes to the methodological advancement

in studying public administration beyond domestic borders.

We Need to Compare, but How? Measurement Equivalence

in Comparative Public Administration

Michael McGuire, Editor

Sebastian Jilke

Erasmus University Rotterdam, the Netherlands

Bart Meuleman

University of Leuven, Belgium

Steven Van de Walle

Erasmus University Rotterdam, the Netherlands

Consider the following survey item: “Overall,

how satisf‌i ed are you with your electricity

supplier? Please give me score from 0 to 10,

where 0 means that you are not satisf‌i ed at all, and 10

means that you are fully satisf‌i ed.”  is is one out of

a battery of items that taps citizen satisfaction with

public services across a wide range of countries.  e

underlying assumption of asking respondents in dif-

ferent national populations the same questions is that

their answers are supposed to be comparable. In other

words, it is assumed that perceptions of what satisfac-

tion means and the way in which people use assigned

scales are equivalent across countries, allowing for

meaningful comparisons. But is the general notion of

what a satisfactory public service is really equivalent

across countries, regions, (groups of) individuals, or

even over time? And are patterns of response styles

the same across dif‌f erent cultures? In this article, we

introduce two major techniques for detecting and

correcting for nonequivalence in the f‌i eld of public

administration, and we show how these methods can

be implemented in applied research.

Comparisons across countries of public administra-

tions, public managers, and interactions and attitudes

of citizens toward government are gaining ground in

public administration research (e.g., Jilke 2014; Kim

et al. 2013; Pollitt and Bouckaert 2011; Van Ryzin

2011).  is is accompanied by an increase in the avail-

ability of cross-national surveys that contain questions