Fragile Snapshot or Stable Relationships?
| Date | 01 March 2012 |
| Published date | 01 March 2012 |
| Author | Lowell W. Barrington |
| DOI | 10.1177/0010414011421311 |
| Subject Matter | Articles |
11BarringtonComparative Political Studies
© The Author(s) 2012
Reprints and permission:
sagepub.com/journalsPermissions.nav
Comparative Political Studies
45(3) 312 –340
Fragile Snapshot or
© The Author(s) 2012
Reprints and permission:
Stable Relationships?
sagepub.com/journalsPermissions.nav
DOI: 10.1177/0010414011421311
http://cps.sagepub.com
What the Orange
and Rose Revolutions
Reveal About the
Stability of Cross-
Sectional Survey Data
Lowell W. Barrington1
Abstract
One of the long-standing criticisms of cross-sectional survey data is that they
provide only a contextually driven “snapshot” of attitudes. These attitudes
are, the “snapshot critique” contends, highly fragile—subject to significant
fluctuation based on events that arise domestically and globally. Although
it makes sense that a major event can alter the percentage of people who
respond to a given survey question in a particular way, it is less obvious
that such an event jeopardizes the validity of multivariate analyses of survey
data collected prior to the event. Given the prevalent use of cross-sectional
survey data in quantitative political research, this question has significant
implications for comparative politics. This study employs survey data from
Ukraine before and after the “Orange Revolution” and from Georgia before
and after the “Rose Revolution.” Its findings challenge the snapshot critique
and support the idea that, even in the wake of a dramatic political event,
the underlying relationships among variables measured by survey data can
remain quite stable.
1Marquette University, Milwaukee, WI, USA
Corresponding Author:
Lowell W. Barrington, Department of Political Science, Marquette University,
478 William Wehr Bldg., Milwaukee, WI 53201-1881
Email: lowell.barrington@marquette.edu
Barrington
313
Keywords
mass attitudes, surveys, cross-sectional data, snapshot, Orange Revolution,
Rose Revolution
It is frequently said that an opinion poll is a snapshot of public opinion. The
problem is that the metaphor is not pushed at all far enough. What needs to be
added is that the snapshot may have been under-exposed or over-exposed; the
lens may have been telescopic where a wide-angle lens was required or vice
versa; the focal length may have been inappropriate; the camera may have
been shaken, and so on. In short, the reality of public opinion must not be
confused with any set of indicators of it.
—Richard Sinnott (1997, p. 9)
In the summer of 2009, Iranians by the thousands took to the streets to protest
what they believed to be a fraudulent presidential election. Encouraged by
presidential candidate and main opposition figure Mir Hossein Moussavi, the
demonstrations centered in the capital of Tehran but popped up in other large
cities across the country. Although the protests were ultimately unsuccessful in
ousting President Mahmoud Ahmadinejad from power, the immediate after-
math of the election in Iran had all the markings of the “color revolutions” that
had taken place earlier in the decade in countries such as Georgia and Ukraine.
Indeed, many called the 2009 events in Iran the “Green Revolution,” from the
color used by Moussavi’s campaign.1
The uprising in Georgia occurred in November 2003, when citizens turned
out to protest an election that, temporarily, kept Georgian President Eduard
Shevardnadze in office. The protests culminated with a flurry of activity on
November 23 and 24. In short order, the “Rose Revolution” led Shevardnadze
to resign and a new election to be scheduled. The subsequent election brought
Mikheil Saakashvili, Shevardnadze’s opponent in the November election, to
power. One year later, the world watched as the Rose Revolution was replayed
on a grander scale in Ukraine. The resulting “Orange Revolution” led to a new
election and victory for Viktor Yushchenko over the previous “winner,” Viktor
Yanukovych. The protests, centered in Kiev but taking place in a number of
cities, involved a sizeable portion of the country’s population.
As in the case of the Green Revolution in Iran, the lasting effects of the
Rose and Orange Revolutions are open to debate (see Kalandadze & Orenstein,
2009).2 Yet they remain defining moments for Georgia and Ukraine. Alongside
the other postcommunist “color” revolutions in Serbia and Kyrgyzstan
(and additional election protests in Armenia, Azerbaijan, and Belarus), they
314
Comparative Political Studies 45(3)
marked a turning point in the semiauthoritarian politics of much of the post-
communist region. Authoritarian and semiauthoritarian leaders could no lon-
ger steal elections without at least some fear of subsequent mass demonstrations
leading to the loss of political power.3
The Rose, Orange, and Green Revolutions are also examples of the kind
of event that critics of quantitative analysis of cross-sectional survey data
point to as a reason one should not trust such data. These data provide, in
the oft-used word of these critics, a “snapshot” of a moment in time, which
is highly contextual and, therefore, highly fragile. Even day-to-day eco-
nomic, social, and political events can alter the views of the population on
a number of important issues; consequently, an event like the color revolu-
tions could be expected to rock mass attitudes as much as or even more than
it does the political system, threatening the validity of data collected prior
to the event.
This study is a test of the “snapshot critique” and an alternative view, the
“stable relationships argument.” The stable relationships argument accepts the
idea that major events like the color revolutions can lead to significant changes
in the summary statistics of a dependent variable—such as the average score
on a scale of attitudes in Ukraine about ties to Russia—but not in the underly-
ing relationships between that dependent variable and other variables in the
model. The snapshot critique, on the other hand, concerns the proposition that
major events like the Orange or Rose Revolutions do likely alter the underly-
ing relationships between the dependent variable—again, such as attitudes
about Russia in Ukraine—and other variables that one might hypothesize to
affect deviations in that variable at the individual level.
This article presents the results of analyses of survey data from 2003 and
2004 in Georgia and 2004 and 2005 in Ukraine to assess the extent to which the
color revolutions in these two countries altered key attitudes, and the relation-
ship between these attitudes and factors such as region of residence, ethnicity,
and language use. The results point to statistically significant changes in atti-
tudes about Russia and the West. But the analyses also reveal a high degree of
stability in the individual-level, multivariate relationships. Although not reject-
ing the snapshot critique outright, taken as a whole, these findings indicate that
researchers should be much more skeptical than they generally are about this
common criticism of cross-sectional survey data.
The Conventional Wisdom of
Cross-Sectional Survey Data as a “Snapshot”
Cross-sectional studies make up a huge portion of scholarly work utiliz-
ing surveys to understand mass attitudes. One of the long-standing criticisms
Barrington
315
of cross-sectional survey data is that they provide only a contextually driven
and highly fragile “snapshot” of attitudes, subject to significant fluctua-
tion based on domestic and global events. It has become widely accepted
across subfields of political science that cross-sectional surveys provide
only a fleeting glimpse of attitudes.4 In an article that otherwise lauds
the contributions of survey research to political science, Brady (2000)
labels the cross-sectional survey as a “snapshot of a moment in time”
(p. 50). Cook and Gronke (2005) concede that their survey from early
2002 “presents a snapshot of opinion at a starkly unusual moment in
American politics” (p. 786), whereas Rueda (2005) acknowledges the
Eurobarometer data he analyzes provide “only a snapshot of individual
preferences” (p. 61).5
This belief leads many scholars to dismiss findings from survey data analy-
sis, including tests of individual-level causal hypotheses involving bivariate or
multivariate relationships. The snapshot critique is a favorite among com-
parativists who emphasize the benefits of qualitative methods over quantita-
tive approaches, many of whom are suspicious of all mass survey data analysis.
They contend that the fragile nature of survey data makes them ultimately
unhelpful in understanding political outcomes. Silverman (1973) summa-
rizes the qualitative critics’ views with the statement that survey data analysis
involves “nothing more than a snapshot taken from a misleading angle and
frozen in time” (p. 185).
Quantitative scholars with data sets containing variables measured over time,
particularly those engaged in panel data research, often join in the qualitative
critics’ attacks. In a work on tolerance in Russia, Gibson (2002) argues that his
panel data allow a more accurate causal story than cross-sectional analyses.
Hilton and Patrick (1970) are more forceful, stating, “Few methodological
generalizations are as widely accepted as the superiority of longitudinal data
over cross-sectional data” (p. 15).
Mainstream political science research methods textbooks and other guides
for those interested in survey research have also come to portray the snapshot
critique as empirical fact, helping...
To continue reading
Request your trial