Argument quality in public deliberations.

• Author: Steffensmeier, Timothy
• Position: Report

ARGUMENT QUALITY IN PUBLIC DELIBERATIONS

Theorists and activists increasingly frame their preferred forms of politics as "deliberative democracy" (Dryzek, 1990, 2000; Gastil & Keith, 2005). Understanding citizen deliberation as political action is neither a novel idea nor exclusive to western political systems. For instance, excavations suggest that the earliest human civilizations had a language to describe people gathering together to make collective decisions. (1) Moreover, ample evidence from ancient Greek city-states support these earlier data (Ober, 1991) and, thereby, reinforce a tradition of politics via citizen deliberations. A precedent for community-based deliberation exists in the United States--during the 1930s, a government funded "American Forum" movement organized community discussions on a national scale (Keith, 2007). These histories, evident in language and organizational structures across cultures and over time, demonstrate a long-standing attraction to deliberative politics.

"Deliberative democracy," a term coined in the 1980s (Dryzek, 2000, p. 2), is the linguistic hook that binds scholarship addressing the normative conditions and efficacy of discursive politics. What deliberation ought to be and what it should yield are questions that have generated a wide range of defining characteristics and outcomes. Barabas (2004) summarizes deliberative ideals as seeking consensus on a contested issue while simultaneously enlightening participants. Consensus and enlightenment align with a Habermasian notion of discourse ethics wherein the ideal speech situation consists of open participation, justification of assertions, consideration of the common good, respect for other participants and a rationally motivated consensus (Habermas, 1985; Steenbergen, Bachtiger, Sporndli, & Steiner, 2003).

Contesting and amending such normative ideals has become something of a cottage industry in scholarship relating to argumentation. Sanders (1997), (2) a staunch skeptic of deliberative democracy, concedes that an outright rejection of deliberative politics seems to be counterintuitive. "A commitment to deliberation is, after all, a commitment to finding a way to address concerns, resolve disagreements, and overcome conflicts by offering arguments supported by reasons to our fellow citizens" (p. 347). Despite the appeal of deliberation, Sanders remains dissatisfied with the current state of deliberative theory-building, primarily because scholars have failed to justify their theories with "substantive or empirical" arguments from ordinary people--the people on whose behalf theorists have argued. These voices are largely absent in creating and testing theoretical claims. Without evidence to the contrary, Sanders holds that when people deliberate they often create counterproductive effects--most specifically, a further privileging of dominant voices. For Sanders, deliberative theorists should refocus their lofty goals and emphasize testimony-laden discussions with an intermediate goal of inclusive participation and mutual respect by the parties to deliberations.

Although deliberative theorists have yet to embrace Sanders's (1997) vision of what ought to replace Habermas's ideal, increasingly they are committed to studying what happens when people attempt to make collective decisions. Steiner et al. (2004) frame contemporary research on deliberative democracy in this manner: "[T]he controversies surrounding the deliberative model cannot be resolved at the level of theoretical speculations and that research needs to go beyond illustrative anecdotes" (p. 42). Steiner et al. acknowledge that empirical research "cannot determine whether deliberation is a good thing in itself" (p. 42), yet it can serve as a "helping hand" in controversies involving democratic theory. Habermas (2006) seemingly concurs, in noting that whether deliberation contributes to "political will-formation and decision-making is, of course, an empirical question" (p. 413).

A call for empirical investigations does not discount the expanding body of research concerning democratic deliberations (see Carpini, Cook, Lomax, & Jacobs, 2004). For example, recent studies have examined ideal deliberative conditions as compared to actual outcomes (Ryfe, 2007). This research includes surveying participants' willingness to participate (McLeod et al., 1999), attitude change (Ackermann & Fishkin, 2002), behavior modification, and influence on public policy. The studies, while important for testing theoretical assertions, are not designed to describe the communicative exchanges that occur during deliberations. As David Ryfe (2006) states, "[W]hat transpires between participants in the forums themselves remains something of a mystery" (p. 73). Ryfe calls for research that focuses on the actual happenings of deliberations--the form(s) of communication in which participants engage in particular. We concur with Ryfe's assessment, in part, because it addresses Sanders's worry that actual deliberations consist of undemocratic practices. We addressed these concerns in a content analysis of forms of discourse occurring in public deliberations of three kinds.

QUALITY OF ARGUMENT

We were most interested in the quality of arguments advanced by participants in deliberations. Our emphasis on quality of argument in a deliberative realm differs from research in persuasion that deals with quality of argument as a variable in models of message processing (Park, Levine, Kingsley-Westerman, Orfgen, & Foregger, 2007; Rains & Mitchell-Turner, 2007). Whereas these studies focus on the persuasiveness of a given message, we were interested in developing a better understanding of the discourse in deliberative forums. In confining our study to deliberative spaces, we focused on a particular form of discourse that functions, in part, on the basis of procedural rules, whether explicitly stated or implicitly practiced.

A procedural account of deliberation-advocated by Habermas (1994) and the pragmadialectical camp (van Eemeren & Grootendorst, 2003)--portrays argument as a vital mechanism for legitimizing democracy (Hicks & Langsdorf, 1999). Hicks and Langsdorf, although sympathetic to the procedural approach, underscore its inadequacy in fully accounting for the constitutive power of deliberation. They suggest that "procedural theories can be strengthened if we acknowledge that procedures not only govern argumentative activity, but constitute those who engage in argumentation" (p. 141). Here, argumentative normativity is a primary concern, both in how procedures reinforce standards and reproduce them among parties to deliberations. By acknowledging that deliberative procedures constitute participants of deliberations, we signal our commitment to understanding the interaction between process (argumentation) and product (argument).

Evaluating the quality of argument involves judging why one argument might be better than another. A distinction between stronger and weaker arguments traditionally runs along lines of giving reasons (Siegel, 1999). In this way, Siegel contends, the stronger argument is one wherein the propositions best justify the claim in a trans-cultural manner (p. 184). Evaluating argument in this way, without accounting for the situated perspectives of arguers, has been challenged by scholars advocating for a pluralistic, culturally bound conception of argument quality (Orr, 1989). This perspective on evaluation accounts for differences (culture, gender, etc.) in both the building and judging of argument.

In our focus on quality of argument in deliberations, we do not aim to settle the epistemic question that is noted above; however, we can help to illuminate interactions between process and product--the interactions that Hicks and Langsdorf contend work to constitute arguers. Investigating the quality of argument from this approach includes more than an evaluation of premises and claims; it includes emphasis on how the product is affected by procedure. To this end, we addressed the following research question: In public deliberations, does the quality of argument vary by the type of deliberative forum? We attempted to answer this question by analyzing the quality of argument in three loci of public deliberation: public hearings, issue forums, and online message boards. Steiner, Bachtiger, Sporndli, and Steenbergen's (2004) Discourse Quality Index (DQI) was the principal instrument providing data for analysis.

DISCLOSURE QUALITY INDEX

Steiner et al. (2004) created the Discourse Quality Index in response to a lack of "quantitative investigations with reliability tests" (p. 53). Although some investigators have analyzed the quality of deliberation in local deliberative forums, they have typically employed qualitative methods. Steiner et al. contend that qualitative studies are important for discovering the subtleties of deliberative practices, but quantitative studies can speak to audiences who prefer different forms of data to be convinced (e.g., economists).

The DQI was formed and tested with a focus on deliberations in formal parliamentary bodies. Underlying the index is the assumption that deliberation occurs on a continuum ranging from "no deliberation" to "ideal deliberation" (Steiner et al., 2004). The DQI categories show the influence of Habermas' discourse ethics in encompassing these principles: 1) open participation in deliberation, which provides participants with space and freedom to advance ideas, with inclusivity and protection against inequality experienced in public discourse as critical values; 2) justification of assertions and claims of validity, which is in line with Habermas' and the DQI's privileging rationality as a condition for healthy deliberation, and signals the importance of reason-giving in weighing decisions; 3) consideration of the common good, or the view that self-interest must be tempered for effective collective decision-making to...