An Empirical Comparison of Seven Populist Attitudes Scales

Published date01 June 2020
Date01 June 2020
DOIhttp://doi.org/10.1177/1065912919833176
Subject MatterArticles
/tmp/tmp-18cOJLSHt41vCt/input
833176PRQXXX10.1177/1065912919833176Political Research QuarterlySilva et al.
research-article2019
Article
Political Research Quarterly
2020, Vol. 73(2) 409 –424
An Empirical Comparison of Seven
© 2019 University of Utah
Article reuse guidelines:
sagepub.com/journals-permissions
Populist Attitudes Scales
https://doi.org/10.1177/1065912919833176
DOI: 10.1177/1065912919833176
journals.sagepub.com/home/prq
Bruno Castanho Silva1 , Sebastian Jungkunz2, Marc Helbling2,
and Levente Littvay3
Abstract
With the recent upsurge of populism in developed and transition democracies, researchers have started measuring
it as an attitude. Several scales have been proposed for this purpose. However, there is little direct comparison
between the available alternatives. Scholars who wish to measure populist attitudes have little information available
to help select the best scale for their purposes. In this article, we directly compare seven populist attitudes scales
from multiple perspectives: conceptual development, questionnaire design, dimensionality, information, cross-national
validity, and external validity. We use original survey data collected online from nine countries in Europe and the
Americas, with around 250 participants per country, in which all seven batteries of questions were present. Results
show that most scales have important methodological and validity limitations in at least one of the dimensions tested,
and should not be used for cross-national comparative research. We recommend populist attitudes items that work
better at capturing populism, and more generally provide guidelines for researchers who want to compare different
scales that supposedly measure the same construct.
Keywords
measurement, populism, structural equation modeling, scale development, psychometric properties
Introduction
questionnaire of the Comparative Study of Electoral
Systems (CSES; Hobolt et al. 2016).1 We collected origi-
Social scientists doing survey research have never had as
nal survey data in nine European and American countries,
much data. Besides well-established cross-national sur-
including all batteries of questions in all samples. We
veys, every week specific surveys are fielded to study
analyze questionnaire design, perform tests with factor
various topics. Two consequences have been the follow-
analysis and Item Response Theory (IRT), and evaluate
ing: (1) survey and questionnaire design become funda-
external validity. At the end, we suggest how scholars can
mental for an ever growing number of scholars, and
make an informed decision as to what scales perform bet-
(2) a proliferation of different ways to measure similar
ter in each aspect, and how that can be expanded beyond
concepts. Examples include authoritarian attitudes populist attitudes studies.2
(Altemeyer 1981; Feldman and Stenner 1997; Oesterreich
2005), ideology (e.g., Lo, Proksch, and Gschwend 2014;
Wilson and Patterson 1968), or political trust (Feldman
Populist Attitudes Defined and
1983; Levi and Stoker 2000; Marien 2011).
Measured
It is often difficult for researchers to decide which
Populism is defined almost identically across all studies
questions, among suggested alternatives, would be better
building on Mudde (2004): a thin-centered ideology
to measure a concept. With the rapid expansion of popu-
according to which society is divided into two homoge-
lism studies, multiple scholars tried measuring populist
neous and antagonistic groups: the “good people” and the
attitudes among individuals, all in different ways. So far,
researchers had little empirical evidence to guide their
1
choice for a battery of questions. In this article, we apply
University of Cologne, Germany
2University of Bamberg, Germany
several psychometric techniques to seven scales measur-
3Central European University, Budapest, Hungary
ing populist attitudes: Akkerman, Mudde, and Zaslove
(2014); Castanho Silva et al. (2018); Elchardus and
Corresponding Author:
Bruno Castanho Silva, University of Cologne, Herbert-Lewin Str. 2,
Spruyt (2016); Oliver and Rahn (2016); Schulz et al.
50931 Köln, Germany.
(2018); Stanley (2011); and the module of the wave 5
Email: bcsilva@wiso.uni-koeln.de

410
Political Research Quarterly 73(2)
“corrupt elites.” Ordinary people are always morally bet-
To capture people-centrism, two scales directly refer
ter than the elite, who illegitimately capture and maintain
to the “will of the people” (Akkerman, Mudde, and
power, betraying people’s interests (Hawkins 2009; Zaslove 2014; Castanho Silva et al. 2018). Others have
Mudde and Rovira Kaltwasser 2013). Broadly, this is the
statements on direct democracy and popular sovereignty,
concept all scales try capturing, with the partial exception
claiming that the people should be consulted for impor-
of Oliver and Rahn (2016), which adds nationalism.
tant policy and political decisions (Akkerman, Mudde,
The most important aspect measured in these projects
and Zaslove 2014; Hobolt et al. 2016; Schulz et al. 2018).
concerns the role ordinary people play in politics and
There are also items gloryfing the “honest” and “hard-
society: the “will of the people” is the highest principle in
working” character of ordinary folk, as well as its com-
a country and needs to be fully implemented into politics
mon sense wisdom (Elchardus and Spruyt 2016; Oliver
(Canovan 1981, 1999). The people should be in charge of
and Rahn 2016; Schulz et al. 2018; Stanley 2011). Some
important decisions and cannot be well represented by
scales focus on the political aspects of people-centrism—
politicians. All studies include items representing popular
popular sovereignty in politics (Akkerman, Mudde, and
sovereignty, the duties of elected politicians toward their
Zaslove 2014; Castanho Silva et al. 2018; Hobolt et al.
electorate, and questions of representative democracy.
2016)—while others give more weight to the romantici-
Another crucial aspect concerns the idea that populists
zation of ordinary people and common sense in general
are against anyone who does not belong to the group of
(Elchardus and Spruyt 2016; Oliver and Rahn 2016).
“ordinary people.” These others form an “elite.” It is not
Two scales combine both aspects (Schulz et al. 2018;
always clear how it is delimited (Jagers and Walgrave
Stanley 2011).
2007, 324), but for most populists, it is the political estab-
When it comes to anti-elitism, all have a strong focus
lishment. Every measure under study here includes items
on government and elected politicians. This brings trou-
on attitudes toward politicians, members of parliament,
ble when applied to countries where populists are in
and the government.
power. Most scales were first tested in countries where
the government was not populist and, therefore, the anti-
Operationalization and Dimensionality
government items worked fine for capturing anti-elitist
views. The increase in the number of populist govern-
Populism is a multidimensional concept, so there are two
ments in Western democracies today makes it necessary
possible approaches to question design. The first is draft-
to rethink the connection between theoretical anti-elitism
ing specific items to capture each dimension sepa-
and its operationalization (Enyedi 2016).
rately—for example, if praise of common people is one
Nonetheless, some include other elite groups, for
dimension, an item should tap exclusively into that with-
instance “experts,” “academics,” or “intellectuals”
out references to the elite. This approach is followed by
(Elchardus and Spruyt 2016; Oliver and Rahn 2016).
Castanho Silva et al. (2018), Oliver and Rahn (2016),
These are a common target for right-wing populists.
Schulz et al. (2018), and Stanley (2011). The second
However, if we think of some cases where left-populists
option is capturing all dimensions simultaneously in a
are heavily popular among, and inspired by, well-edu-
single scale, including single items that refer to two or
cated individuals—Podemos in Spain being a case in
three dimensions in themselves. That is seen in point—then these terms will hardly determine a credible
Akkerman, Mudde, and Zaslove (2014); Elchardus and
elite for populist supporters. Broader concepts are also
Spruyt (2016); and the CSES.3
used, such as “interest groups” (Akkerman, Mudde, and
With the exception of Elchardus and Spruyt (2016),
Zaslove 2014), “the system” and “the powerful” (Oliver
all projects differentiate between three different subcom-
and Rahn 2016), or “the big interests” (Castanho Silva
ponents, which cover similar dimensions and can be sub-
et al. 2018). Ultimately, these might be more generaliz-
sumed under the terms of people-centrism, anti-elitism,
able across different types and contexts of populism to
and anti-pluralism. Oliver and Rahn (2016), for exam-
capture the essence of anti-elitist feelings beyond anti-
ple, have the dimension “mistrust of experts” besides
government expressions.
“anti-elitism,” including several questions on the role
Importantly, all scales measure populism independent
ordinary people play in society. There, the common
of the so-called “host ideology,” that is, they do not try to
sense knowledge of ordinary folk is played up as supe-
capture specifically right- or left-wing populism.
rior to experts’ ideas, which is one way that populists
Theoretically, this follows the definition laid out above
praise common people....

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT