Multilingual Sentiment Analysis: A New Approach to Measuring Conflict in Legislative Speeches

• Author: Stuart Soroka,Jens Wäckerle,Will Lowe,Sven‐Oliver Proksch
• DOI: http://doi.org/10.1111/lsq.12218
• Published date: 01 February 2019
• Date: 01 February 2019

97

LEGISLATIVE STUDIES QUARTERLY, 44, 1, February 2019

DOI: 10.1111/lsq.12218

SVEN-OLIVER PROKSCH

University of Cologne

WILL LOWE

Princeton University

JENS WÄCKERLE

University of Cologne

ST UART S OROKA

University of Michigan

Multilingual Sentiment Analysis: A

New Approach to Measuring Conflict

in Legislative Speeches

Comparative scholars of legislative politics continue to face the challenge

of measuring a key theoretical concept: conflict at the level of legislative bills. We

address this challenge with a multilingual sentiment-based approach and show

that such a measure can effectively capture different types of parliamentary con-

flict. We also demonstrate that an automated translation of the dictionary yields

valid results and therefore greatly facilitates comparative work on legislatures.

Our applications show that a sentiment approach recovers government-oppo-

sition dynamics in various settings. The use of a simple, translatable sentiment

dictionary opens up the possibility of studying legislative conflict in bill debates

across languages and countries.

Despite the advent of text-as-data tools and easy access to

electronic parliamentary records, comparative scholars face an

unresolved challenge: measurement of legislative conflict across

languages. Theories of legislative politics often predict political

outcomes as a function of conflict between government and op-

position parties or within government coalitions on specific is-

sues, but existing me asures are typically h ighly agg regated (at the

policy dimension level). Text-as-data approaches promise to en-

able the estimation of issue-specific conflict, but by eschewing

comparative human policy judgments to focus on comparative

word usage, they make cross-national comparisons difficult due

to unresolved bridging problems.

© 2018 Washing ton University in St. Louis

98Sven-Oliver Proksch et al.

We take a step towards resolving this problem. We present

a multilingual sentiment-based approach to estimating legisla-

tive conflict and show that this measure can effectively capture

government-opposition conf lict in various settings. We valid ate

the approach using debates in the European Parliament and

demonstrate that automated dictionary translations provided

by Google offer a reasonable basis for sentiment analysis in

different languages. Automated sentiment measures correlate

highly with manual coding for almost all translated dictionar-

ies. We also show that using a translated dictionary on an orig i-

nal speech corpus yields the same results as using an original

dictionary on a translated speech corpus, but at a significantly

lower cost to the researcher. Finally, we show that human cor-

rections of the translated dictionaries nevertheless yield some

improvements.

We apply the sentiment dictionaries to several settings. First,

we show that a sentiment approach provides valid results for

measuring individual MP dissent in budget debates in Ireland.

Second, we demonstrate that opposition sentiment predicts the

occurrence of consensus on government bills in Germany. For

both the Irish and Ge rman settings, we furthe rmore confirm that

government MPs and cabinet me mbers expres s, on average, more

positive sentiment on government bills than opposition MPs.

Finally, using a cross-national parliamentary debate corpus, we

demonstrate that sentiment always recovers the differences be-

tween government and oppos ition and that this measure is related

to important ideological, electoral, and institutional features of

legislative debate.

We take each of these tests a s evidence of the potential for the

use of sentiment tools in estimating legislative conflict. The main

advantage of the use of senti ment is not just simply to identify leg-

islative conflict, however—other tools are able to do this as well.

The use of a relatively simple, translatable sentiment tool opens

up the possibility of facilitating the analysis of legislative conf lict

across languages and countries. This is the main objective of the

work that follows. We begin w ith a review of the state of the field.

Challenges of Text-as-Data in Legislative Politics

While text-as-data approaches have become a common fea-

ture in legislative politics (e.g., Diermeier et al. 2012; Hillard,

99Multilingual Sentiment Analysis

Purpura, and Wilkerson 2008; Hopkins and King 2010; Laver,

Benoit, and Garry 2 003; Lowe 2008; Quinn et al. 2010; Slapin and

Proksch 2008), applications are inherently limited to unilingual

studies. While there is certainly a growing interest in analyzing

multilingu al corpora (Lucas et al. 2015; Proksch and Slapin 2010),

conventional approaches are limited to an application i n one lan-

guage. Moreover, existing research does not offer solutions as to

how such approaches might be applied in other languages and,

more importantly, how measures extra cted from text can be com-

pared across languages.

One important unresolved challenge is the measurement of

issue-specific legislative conflict. For instance, the literature on

coalition governments predicts various implications of coalition

divisiveness on policy issues: Bills on which coalition members

are divided ta ke longer to be introduced on the legislative agenda

(Martin 200 4; Zubek and Klüver 2015); once introduced they tend

to take longer to pass (Martin and Vanberg 2011); and they are

subject to greater s crutiny in parliament (Fortunato, Marti n, and

Vanberg 2017; Martin and Vanberg 2005, 2011, 2014; Pedrazzani

and Zucchini 2013). Similarly, regarding the implementation of

EU law in member states, coalition governments should be more

likely to resort to the use of a legislative instr ument—rather than

an administrative procedure —when coalition parties are divided

on an issue (Franchino and Høyland 2009). With regard to the

conflict between government and opposition, the literature on

minority governments suggests that cabinets make policy con-

cessions to the opposition in exchange for votes on specific bills

(Strom 1990). Even in majority settings, t here is growing evidence

that policymaking in parliament may be more consensual be-

tween government and opposition on more bills than previously

thought (Andeweg 2013; De Giorgi and Marangoni 2015).

It is notable that while the theoretical literature operates

at the level of individual policy issue (e.g., legislative bill), most

measures used in empirical analyses are highly aggregated or

not specific to the legislative context. This includes measures re-

lated to left-right ideology from the Manifesto Project (Budge et

al. 2001; Budge, Robertson, and Hearl 1987; Klingemann et al.

2006; Laver and Budge 1992) or measures on policy dimensions

from expert surveys (e.g., Bakker et al. 2015; Benoit and Laver

2006; Hooghe et al. 2010; Laver and Hunt 1992). For comparative

legislative scholars, this necessitates a compromise: Researchers

must often accept a variable that is measured at the wrong level