Ballot Question Readability and Roll-Off: The Impact of Language Complexity

• Author: Shauna Reilly,Sean Richey
• Published date: 01 March 2011
• DOI: 10.1177/1065912909349629
• Date: 01 March 2011
• Subject Matter: Articles

Political Research Quarterly

64(1) 59 –67

© 2011 University of Utah

Reprints and permission:

sagepub.com/journalsPermissions.nav

DOI: 10.1177/1065912909349629

http://prq.sagepub.com

Ballot Question Readability

and Roll-Off: The Impact

of Language Complexity

Shauna Reilly1 and Sean Richey1

Abstract

Ballot questions often feature obscure and legalistic language that is difficult to comprehend. Because the language of

ballot questions is often unclear, the authors hypothesize that questions with lower readability will have higher roll-off

because voters will not answer questions they do not understand. The authors use an objective measure of readability

to code readability scores for 1,211 state-level ballot questions from 1997 to 2007. Using hierarchical linear regression

models of state-level data, the authors find that increased complexity leads to more roll-off. The authors further

analyze some possible influences on readability by examining whether it is affected by the question topic.

Keywords

direct democracy, elections, ballot language, participation

Does the complex language of ballot questions lead to

roll-off? Direct democracy propositions are far more com-

plex than traditional candidate vote decisions. These

propositions ask the public about a variety of policies with

few heuristics such as party identification or incumbency.

In addition to these cognitive difficulties, they often fea-

ture obscure and legalistic language that is difficult to

comprehend. The complex wording may explain why vot-

ing on ballot measures is often less than voting for higher

offices on the same ballot, because people skip the mea-

sures they do not understand. We examine this potential

impact on roll-off by using a quantifiable objective mea-

sure of language readability to determine the complexity

of ballot question wording. To evaluate if readability

influences roll-off, we coded readability scores for all

state-level ballot questions from 1997 to 2007 (1,211 bal-

lot measures). Using hierarchical linear regression models

of state-level data while controlling for other known

determinants, we find that less readability leads to more

roll-off. We further analyze what influences ballot ques-

tions to be written in a less readable way, and whether

readability is affected by question topic.

Prior research has repeatedly shown that complex

wording for survey questions increases nonresponse

bias (e.g., Mondak 1994). Using this evidence, we expect

poor readability to also increase nonresponse to ballot

questions, known as roll-off. We find that the average

readability of these questions is far above the reading lev-

els of average citizens. By gathering the wording for each

ballot measure and putting them through a Flesch-Kincaid

Grade Level readability test, it is possible to ascertain at

what grade level the question is written. For example, the

median resident of the state of Georgia reads at an eighth-

grade level. We find that the average Georgia ballot ques-

tion is written so that understanding it requires someone

to read at the equivalent of a twenty-second-grade level.

This language complexity far surpasses a plausible level

of understanding by the public.

This research informs our understanding of direct

democracy, as it examines in a new way the classic ques-

tion of whether voters are prepared to directly make laws.

If a major portion of the electorate cannot understand

the language of the question, then this seems to limit the

potential for a proper vote choice. It is important to note

that voters may choose not to answer—roll-off—or they

may guess at what the proposition means, which may

very well lead to a nonoptimal vote choice. Testing this

possibility is beyond the scope of this research project,

but it shows the importance of writing readable ballot

questions.

1Georgia State University, Atlanta, GA, USA

Corresponding Author:

Sean Richey, Georgia State University, Department of Political

Science, 38 Peachtree Center Ave., Suite 1005, Atlanta, GA 30303,

USA; phone: (404) 413-6152

E-mail: srichey@gsu.edu