Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart.

• Author: Cheng, Edward K.
• Position: Book review

SUPER CRUNCHERS: WHY THINKING-BY-NUMBERS Is THE NEW WAY TO BE SMART. By Ian Ayres. New York: Bantam Dell. 2007. Pp. 272. $25.

INTRODUCTION

The quants are coming! And they are here to stay--so argues Professor Ian Ayres (1) in his new book, Super Crunchers, which details the brave new world of statistical prediction and how it has already begun to affect our lives. For years, academic researchers have known about the considerable and at times surprising advantages of statistical models over the considered judgments of experienced clinicians and experts. Today, these models are emerging all over the landscape. Whether the field is wine, baseball, medicine, or consumer relations, they are vying against traditional experts for control over how we make decisions.

To be sure, given its intended popular audience, Super Crunchers does not push the envelope in the judgment and decision-making field, spending most of its effort on summarizing and "translating" the recent successes in statistics and econometrics for the lay reader. But in this endeavor, it succeeds. Those who ordinarily cringe at numbers will find the book a delightful and educationally worthwhile read.

For the legal system, the take-home of Ayres's book and the examples he describes is clear. Courts should be using more statistical decision rules, not only because they promise greater accuracy, but also because they provide the consistency and transparency to which the law often aspires. In line with the Supreme Court's contemporary pronouncements on scientific evidence, (2) courts should be skeptical of traditional experts who testify from personal experience and intuition without quantified empirical data, and be more accepting of statistical evidence.

Although Ayres may have originally wanted to entitle his book The End of Intuition (pp. 55-56), the reality may be far more complicated than this suggests. A substantial recent literature has developed showing the superiority of intuitions and gut feelings, which the pro-statistics crowd has effectively ignored. In addition, supercrunching raises deep issues about profiling, individualized justice, and the nature of legal practice. Navigating these tensions between intuition, statistics, and the law will be the key challenge for the future.

1. BRAVE NEW WORLD

   Super Crunchers begins with an entertaining introduction to statistical modeling through a series of memorable examples. At the outset, we meet Orley Ashenfelter, the Princeton professor whose regression model for Bordeaux wine quality and prices sparked outrage among wine connoisseurs over a decade ago (pp. 1-6). Although famed wine taster Robert Parker denounced Ashenfelter as "an absolute total sham" (p. 3), Ashenfelter's simple regression model involving only rainfall and temperature data performs remarkably well in predicting wine quality--most importantly, his model predicts better than the experts. (3)

   Wine tasters are not the only traditional experts under siege. Baseball fans will appreciate Ayres's reference to Michael Lewis's Moneyball. (4) The time-honored baseball scout, who spends countless hours watching prospects and assessing if they have "what it takes," now competes against the statistical model. The highly publicized success of "sabermetrics" for the Oakland A's and Boston Red Sox has made supercrunching a permanent fixture in America's pastime (pp. 8-9).

   Similarly, the clinical judgment of physicians is under increasing attack, as seen in the trend toward evidence-based medicine (pp. 81-102). Doctors unsurprisingly fall prey to the same mental biases that psychologists have shown to afflict the rest of us: They are overly impressed by anecdotal evidence, even though such reasoning can lead to incorrect inferences based on coincidence (p. 89). Once they formulate a theory or diagnosis, they are susceptible to tunnel vision, failing to consider alternatives and ignoring contradictory evidence. (5) Sometimes, the medical profession does not even know why a particular procedure is performed a certain way--it is merely how things have always been done. (6) Those familiar with recent debates over scientific evidence in forensics and toxic torts will undoubtedly recognize the tune. (7)

   Statistical tools offer a path out of this mess. For example, computer decision-tools can help doctors' intuition by suggesting alternative diagnoses (pp. 98-99). Digitalization and aggregation of medical records means more data for epidemiological analyses, and the possibility of computer-based decision-making is not far off, presuming that the medical establishment allows it (pp. 99-100).

   But the transformative potential of statistical models does not end at replacing traditional experts like wine connoisseurs, baseball scouts, and medical doctors. Ayres shows that number crunching is changing social and business interactions as well. Film companies use statistical tools to predict blockbusters (pp. 144-49). Matchmaking, a task traditionally reserved to friends and relatives, has become a sophisticated, statistically driven industry (pp. 23-28). Statistical models are even revolutionizing consumer relations. Credit card companies can lower interest rates, airlines can dole out flight perks, and casinos can swoop in with palliatives to keep their "best" customers placated just enough (pp. 28-31, 47-50, 58-60). As Ayres playfully suggests, in the future, when a seller starts giving you freebies, watch out (pp. 172-73).

   At approximately its midpoint, Super Crunchers turns to cover some well-trodden ground in the decision-making literature that shows statistical methods to be often more accurate than experts. (8) One such study that Ayres discusses (p. 111) is a comprehensive meta-analysis of the clinical-statistical literature by psychologist William Grove and others, in which out "[o]f the 136 studies, 64 favored the actuary[,] ... 64 showed approximately equivalent accuracy, and 8 favored the clinician." (9) Indeed, in some of these studies, statistical models were superior despite the experts being privy to more information (statistical models generally require a shockingly small number of factors) (10) and even more outrageously, despite experts having the model results at their disposal (pp. 116-17). Having a human override for catching "stupid" machine errors turns out to be counterproductive, because the safety valve ends up introducing more errors than it prevents (pp. 121-23).

   Ayres also touches on the broader implications of statistical decision making and suggests some reasons for society's resistance to statistical methods. One reason is what Ayres entitles "The Status Squeeze" (p. 166). By standardizing the decision-making process, statistical models "threaten[] the status and respectability of many traditional jobs," particularly those of professionals with expertise (p. 166). As Ayres recounts, loan officers traditionally held "moderately high status position[s,] ... were well paid and had real power to decide who did and did not qualify for loans" (p. 166). Statistical models, however, have since gutted the loan officer position, making them "nothing more than glorified secretaries" (p. 167). The loan officer tale is in many ways an ominous one for doctors and other professionals watching statistical models erode away their discretion. Their prestige, autonomy, and livelihood are at risk as the quants encroach on their territory.

   Super Crunchers also briefly touches on a few other concerns about statistical methods: privacy, discrimination, and error. In terms of privacy, supercrunching certainly makes it more difficult for people to escape their pasts and futures, but as Ayres...