Political Attacks in 280 Characters or Less: A New Tool for the Automated Classification of Campaign Negativity on Social Media

Published date01 May 2022
Date01 May 2022
AuthorVladislav Petkevic,Alessandro Nai
Subject MatterArticles
American Politics Research
2022, Vol. 50(3) 279302
© The Author(s) 2021
Article reuse guidelines:
DOI: 10.1177/1532673X211055676
Political Attacks in 280 Characters or Less: A
New Tool for the Automated Classif‌ication of
Campaign Negativity on Social Media
Vladislav Petkevic
,and Alessandro Nai
Negativity in election campaign matters. To what extent can the content of social media posts provide a reliable indicator of
candidatescampaign negativity? We introduce and critically assess an automated classif‌ication procedure that we trained to
annotate more than 16,000 tweets of candidates competing in the 2018 Senate Midterms. The algorithm is able to identify the
presence of political attacks (both in general, and specif‌ically for character and policy attacks) and incivility. Due to the nov el
nature of the instrument, the article discusses the external and convergent validity of these measures. Results suggest that
automated classif‌ications are able to provide reliable measurements of campaign negativity. Triangulations with independent
data show that our automatic classif‌ication is strongly associated with the expertsperceptions of the candidates campaign.
Furthermore, variations in our measures of negativity can be explained by theoretically relevant factors at the candidate and
context levels (e.g., incumbency status and candidate gender); theoretically meaningful trends are also found when replicating
the analysis using tweets for the 2020 Senate election, coded using the automated classif‌ier developed for 2018. The implications
of such results for the automated coding of campaign negativity in social media are discussed.
negative campaigning, US Midterms, machine learning, neural networks, incivility
Modern politics is a hard-fought business. The public is
increasingly hostile toward those they consider as their rivals
(Iyengar et al., 2012;Iyengar & Westwood, 2015), antago-
nistic and aggressive political f‌igures are on the rise across the
globe (Nai & Martinez i Coma, 2019a), and attacks seem at
times the very essence of election campaigns (Ansolabehere
et al., 1994;Lau & Pomper, 2004). To be sure, negativity in
politics matters. Negative information, when compared to
equivalent positiveinformation, is more likely to be seen,
processed, and remembered (e.g., Rozin & Royzman, 2001).
Also because of that, some scholars show that negative
messages can convey important and useful information to the
voters, promote issue knowledge, cue the voters that the
election is salient, and thus worth the emotional and cognitive
investment, and ultimately stimulate the interest of the public
(Finkel & Geer, 1998;Martin, 2004;Geer, 2006). On the
other hand, however, strong evidence suggests that negative
campaigning can be a detrimental force in modern democ-
racies. Negative and harsh campaigns can reduce turnout and
political mobilization, depress civic attitudes such as political
eff‌icacy and trust, foster apathy, and generally produce a
gloomierpublic mood (Ansolabehere et al., 1994;Thorson
et al., 2000;Yoon et al., 2005). On top of this, a case can be
made that decreased trust in the political game and increased
political cynicism are likely to reinforce the consolidation of
antagonistic and disruptive movements, which often feed
off the public discontent. Whether a positive or detrimen tal
force, few would contest that negativity is a key component of
contemporary electoral democracies.
In recent years, the dynamics of electoral campaigning
have been reshaped by the emergence of social media
(Gainous & Wagner, 2014;Graham et al., 2016;Straus et al.,
2013). Online communication, especially via social media,
allows political actors to cut the middlemen”—for instance,
journalistic gatekeepingand communicate directly with
their audience, in what is often referred to as one-step f‌low of
Faculty of Social and Behavioral Sciences, University of Amsterdam,
Amsterdam, Netherlands
Amsterdam School of Communication Research (ASCoR), University of
Amsterdam, Amsterdam, Netherlands
Corresponding Author:
Vladislav Petkevic, Faculty of Social and Behavioral Sciences, University of
Amsterdam, Nieuwe Achtergracht 166, 1018 WV, Amsterdam, Netherlands.
Email: v.petkevich@uva.nl
communication(Bennett & Manheim, 2006). Such facili-
tated access to the people is one of the reasons why online
communication is particularly favored by populists (Engesser
et al., 2017). In recent years, several studies have assessed the
presence of negativity in social media (e.g., Auter & Fine,
2016;Ceron & dAdda, 2016;Evans et al., 2014;Evans &
Clark, 2016;Gainous & Wagner, 2014;Gross & Johnson,
2016). Broadly speaking, these studies f‌ind conf‌irmation that
the main trends of strategic campaigning found for traditional
techniquesfor instance, that challengers tend to attack more
than incumbents (Gainous & Wagner, 2014)are also found
when looking at campaigning on social media. Those existing
studies tended to rely on manual coding of social media posts.
For instance, in their analysis of the drivers of negativity of
Facebook during the 2010 Midterms, Auter and Fine (2016)
manually coded more than 14,000 posts. Similarly, Ceron and
dAdda (2016) hand-coded more than 15,000 Tweets pub-
lished by competing candidates prior to the 2013 Italian
general election. Recent advances of machine learning ap-
proaches have made it increasingly affordable to dive into
very large amounts of data, which were inaccessibleor
required time-intensive coding effortsup to recently due to
insuff‌icient computational power and the preference for
manual coding. In this article, we expand the growing lit-
erature on automated classif‌ication of textual data within the
context of political communication. We introduce a neural
network classif‌ier that we trained to automatically annotate
the tweets of candidates competing during the 2018 US
Senate Midterms elections, in terms of the presence of po-
litical attacks. The algorithm was run on approximately
16,000 tweets, posted by 63 candidates for the period be-
tween September 1
and November 6
, 2018 (the day of the
election). After presenting the results of the classif‌ication, the
article will test the external and convergent validity of the
measure; more specif‌ically, we will check whether the results
make sense in terms of factors that can be theoretically ex-
pected to drive the presence of negativity in the candidates
tweets (e.g., the incumbency status of the candidate), and by
triangulating the measure with an independent dataset about
the content of candidatescampaigns in the 2018 Midterms,
as assessed by expert surveys (Nai & Maier, 2020).
The reason for studying the 2018 midterm elections was
two-fold. Firstly, from a conceptual standpoint, the US senate
elections provide a unique opportunity to study a series of
extremely similar elections with a reduced number of com-
petitors, happening simultaneously within the same broad
societal, cultural and, ultimately, political context (Lau &
Pomper, 2001,2004)while, at the same time, being able to
control for the most important differences at the contextual
level (e.g., how close the race was). Yet, even if driven by
state-level dynamics, Senate Midterms elections all partici-
pate to the broader national context and political dynamics.
The 2018 Midterms were not an exception in this sense, and
the results in each state had fundamental national implications
in terms of, for example, the control of the upper house (so
central to the recent dynamics of presidential impeachment of
Donald J. Trump). In other terms, the Senate Midterms are an
ideal research setting, allowing all the benef‌its of variation
both at the candidate and context levelswhile keeping most
of the broad cultural and political dynamics, assumed to be
shared across all state-level elections at bay, so to speak.
Indeed, especially compared with Presidential elections,
Senate elections can be seen as methodologically superior
for the study of campaign dynamics (Lau & Pomper, 2004,p.
6). Secondly, the 2018 Midterm Senate elections provide us
with the unique opportunity to test the convergent validity of
our data by comparing it with other, independent data about
the same elections and the same dynamics (i.e., how negative
the candidates in the Midterms went against each other).
More specif‌ically, we will triangulate the tone of the can-
didatescampaign on Twitter with expert ratings provided by
independent scholars (Nai & Maier, 2020).
The rest of this article proceeds as follows. In section 2,we
describe the empirical procedure employed to develop the
algorithm for the automated coding of the negativity in
tweets. Section 3 then presents three tests. First, we test the
convergent validity of the algorithm, by comparing it with the
measure of negativity from independent data using expert
surveys. Second, we test its external validity by checking our
measurement against some trends that can be theoretically
expected (i.e., the fact that challengers should be expected to
be more likely to go negative, or that female candidates tend
to use gentler campaigns. Finally, third, we investigate
whether applying the coding algorithm to a different set of
datathe campaign on Twitter during the 2020 Senate
electionyields results that are also theoretically valid. As
we will see, our algorithm scores well in both external and
convergent validity, suggesting that the automated coding of
social media posts is an effective alternative to standard
measurement of campaign content. The last section concludes
the discussion and glimpses over the directions of future
Supporting materials for this article are available at the
following Open Science Foundation repository: https://osf.io/
up826/. The repository includes (i) the Jupyter Notebook
(Python) f‌ile with the code that was used to pre-process the
raw data, build the classif‌ier, and annotate the whole dataset,
(ii) the annotated dataset of all tweets, (iii) the archive with
the text of all the collected tweets, and (iv) an excel f‌ile with
the reliability assessment of the initial sample of tweets coded
(see below).
Measuring Negative Campaigning in Tweets
Data and Procedure
During the 2018 Midterms, 33 Class 1 Senate seats were up
for grabs (plus additional special elections in Minnesota and
Mississippi to f‌ill vacancies, but which we will not analyze
here); Democrats were holding 26 of these seats, and
280 American Politics Research 50(3)

To continue reading

Request your trial

VLEX uses login cookies to provide you with a better browsing experience. If you click on 'Accept' or continue browsing this site we consider that you accept our cookie policy. ACCEPT