Does the readability of your brief affect your chance of winning an appeal? An analysis of readability in appellate briefs and its correlation with success on appeal.

• Author: Long, Lance N.

1. INTRODUCTION

   The short answer is "no"--at least if by "readability" you mean readability as judged by two of the several well-recognized readability formulas developed by researchers during the past fifty or sixty years. (1) Using the Flesch Reading Ease scale and the Flesch-Kincaid Grade-Level scale, we analyzed the readability of 882 state, federal, and United States Supreme Court briefs and found no statistically significant relationship between the readability of those briefs and success on appeal. This is good news for those who would like to believe that appeals are decided on the merits of a case and that the success of an appeal is not influenced by the "readability" of a brief (or by any other writing convention, for that matter). It is also good news for critics of readability formulas, who marc believe either that such formulas are fundamentally flawed (2) or that such formulas have little to do with legal writing. (3) It provides readability critics with yet another argument for disregarding readability analyses (even if readability is valid, it doesn't make a quantifiable difference). On the other hand, this may be less welcome news for legal writing professionals who may want to believe that the likelihood of success on appeal can be increased by writing a more "readable" brief and that a computerized readability formula can provide a basis for determining readability. (4)

   Our study was an attempt to determine whether using long sentences and long words correlates with success on appeal. The Flesch readability formulas that we used in our study measure precisely those two elements. We were not particularly concerned about whether readability formulas accurately measure the appropriateness of a particular text for a particular reader. Appellate briefs, no matter how readable or unreadable they may be, are read by a highly educated audience. But for all readers, including highly educated readers, it is generally easier to read shorter sentences and shorter words. (5) Could this possibly make a difference in the outcome of an appeal?

   Our study suggests that the length of sentences and words, which is "readability" for our purposes, probably does not make much difference in appellate brief writing. First, we found that most briefs are written at about the same level of readability; there simply is not much difference in how lawyers write appellate briefs when it comes to the length of sentences and words. Furthermore, the readability of most appellate briefs is well within the reading ability of the highly educated audience of appellate judges and justices. Second, the relatively small differences in readability are not related to the outcome of an appeal in a statistically significant manner. Our study did show, however, that the opinions of judges and justices are less readable than lawyers' briefs and that the opinions of dissenting judges or justices are the least readable of all the appellate writing we analyzed. Ultimately, we conclude that readability, as determined by the Flesch Reading Ease scale, is a non-issue for legal writing at the appellate level.

   The analysis discussed in this Article uses a methodology and approach similar to that used in the authors' previously published article, (6) which described "an empirical study of 800 federal and state appellate briefs randomly selected for the purpose of determining whether any relationship exists between intensifier use in the parties' briefs and the outcomes in those cases." (7) This article utilizes a random selection of those same 800 briefs and adds to them the petitioner and respondent briefs written to the United States Supreme Court in every case in which the Court issued an opinion from the time Justice Alito joined the Court until shortly before the retirement of Justice Souter.

   A readability study using the Flesch Reading Ease scale and the Flesch-Kincaid Grade scale was performed on the Supreme Court briefs and the randomly selected state and federal briefs, as well as the court opinions associated with those briefs. This Article examines the results of that analysis.

   Although readability did not appear to be related to outcome, there was a statistically significant relationship between the readability of the courts' majority and dissenting opinions. Dissenting opinions are decidedly less readable than majority opinions.

   Part II of this Article discusses readability formulas generally and the Flesch Reading Ease scale and the Flesch-Kincaid Grade scale specifically, including the criticisms and limitations of readability formulas. Part III explains our analysis methodology and the results of our analysis. Part IV contains a discussion of our results and our conclusions.

2. READABILITY, READABILITY FORMULAS, AND LEGAL WRITING

   It is beyond the scope of this article to fully examine and explain readability formulas or the history and theory behind those formulas. (9) A short summary of the concept of readability, however, is provided here to explain the purpose of readability formulas and to explain why we applied one such formula to appellate briefs and opinions.

   "'Readability' is what makes some texts easier to read than others." (10) Since the 1920s, researchers, including linguists, educators, psychologists, and other scholars, have analyzed writing to determine what makes it more or less readable. (11) By the 1950s, several formulas for assessing readability had been developed, (12) and "[b]y the 1980s, there were 200 formulas and over a thousand studies published on the readability formulas attesting to their strong theoretical and statistical validity." (13)

   Each of the hundreds of readability formulas uses a different set of semantic and syntactic factors to determine readability, but the most frequently used factors are word complexity and sentence length. (14) And while these rather simple "surface features" exclude any consideration of content, grammar, or organization, over fifty years of research have shown that these factors are the best predictors of readability based on comprehension tests that do consider content, grammar, and organization. (15) Although almost every conceivable linguistic factor has been included in the scores of different formulas, and some formulas include a dozen or more factors, the addition of more factors does little to increase the accuracy of readability predictions and renders the formulas much more difficult to use. (16) "Put another way, counting more things does not make [a] formula any more predictive of reading ease but takes a lot more effort." (17)

   Of the many readability formulas, some of the more popular and accurate formulas that rely on sentence and word length include the SMOG (Simple Measure of Gobbledegook) formula, which "measures the number of words of more than 2 syllables in a sample of 30 words," (18) the Gunning Fog Index, which uses two variables: "in a sample of 100 words, the average number of words per sentence and the number of words of more than 2 syllables," (19) the Flesch Reading Ease formula, which measures "the number of syllables and the number of sentences for each 100-word sample," and the Flesch-Kincaid Grade Level formula, which measures the same variables, but converts them to a grade level calculation. (20) Flesch Reading Ease scores range from 0 to 100; a score of 0 is practically unreadable, a score of 30 means the reading is "very difficult," a score of 70 means the reading is suitable for adult audiences, and a score of 100 means the reading is easy (21) and should be readable by someone with a fourth grade education who is "barely 'functionally literate.'" (22)

   These formulas are popular because they are relatively easy to use (all four can be applied with readily available software), (23) and they appear to be accurate because they correlate well with more sophisticated, content-based measures of reading comprehension. (24) The Flesch Reading Ease formula is probably the most influential and popular readability formula, due in part to its adoption by Microsoft Word. (25) The Flesch Reading Ease formula uses the following formula to determine readability: "Reading ease (RE) = 206.835 - 84.6s - 1.015w where s = the average number of syllables per word and w = the average number of words per sentence." (26) The Flesch-Kincaid Grade Level formula uses the following formula to determine readability: Grade level (GL) = .39s + 11.8w - 15.59. (27) Therefore, our analysis considers only the relationship of word length and sentence length with success on appeal.

   Even though readability formulas correlate with reading comprehension, this correlation has been questioned, and recently, the reliability of readability formulas has been criticized. (28) Some scholars and experts claim that readability depends more on the literacy, motivation, and background of the reader than the surface factors of the text. (29) Some further claim that reliance on such formulas can actually decrease the readability of text, especially when "writers ... write to the formulas." (30) By trying to lower reading difficulty through the use of shorter sentences, a writer can...