What Even Is a Criminal Attitude?--And Other Problems with Attitude and Associational Factors in Criminal Risk Assessment.

• Author: Karp, Beth

Introduction A. Background. B. Article Overview I. Overview of Select Risk Assessment Instruments A. Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) B. Level of Service Inventory-Revised (LSI-R) C. Ohio Risk Assessment System (ORAS) and Its Derivatives D. Offender Screening Tool (OST) and Modified Offender Screening Tool (MOST) 1. OST and MOST content--known and inferred 2. OST and MOST design problems II. How Risk-Needs Quantification Works and Doesn't Work A. How Risk is Measured 1. Typical risk assessment factors 2. Commonly used statistics: AUC and r a. AUC b. Correlation coefficients (r) B. No One Has a Good Handle on How to Assess Criminogenic Needs 1. AUC and r tell an incomplete story 2. The Gendreau meta-analysis is not an oracle 3. Dynamic factors are only weakly correlated with crime 4. Call for item-level transparency C. More Factors, More Problems 1. Attitude and associational factors do not improve predictive accuracy 2. "Need" factors should be explained, not quantified III. Speech and Expressive Association A. Abstract Beliefs are Inadmissible at Sentencing Under the First Amendment 1. Dawson requires a nexus 2. Factors in general risk assessment instruments lack a nexus to the defendant's offense 3. "It's Just One Factor" does not cure First Amendment harms B. Normative Issues in Attitude Assessment 1. What even is a criminal attitude? 2. Attitudes as racial proxies 3. Attitude measurement problems IV. Peer and Family Relationships A. Risk Assessment Associational Items 1. Criminal peers 2. Family criminal history B. Constitutional Protections for Peer and Family Relationships 1. The substantive due process dead end a. The Jaycees quagmire b. Peer association items do not violate substantive due process 2. The First Amendment potential pathway 3. The Equal Protection proscription of kin punishment C. Echoes of Eugenics in Criminal Risk Assessment D. Garbage-In-Garbage-Out-Garbage-In-Garbage-Out (Peer Association Measurement Problems) V. Beware the Shiny Legal Penny Conclusion Introduction

1. Background

   Risk assessment instruments have become pervasive in American criminal justice, heralded as a means to reduce mass incarceration, protect the public, conserve resources, and properly individualize sentencing. (1) The instruments are intended to supplement human discretion in criminal justice decisionmaking (2) by taking a set of quantitative inputs, like number of prior convictions or level of educational attainment, (3) and attempting to predict the risk of outcomes like recidivism or failure to appear. (4) Their most well-known and controversial usage is in pretrial release determinations, but they are employed at every phase of criminal justice proceedings, including sentencing, prison assignment, and parole. (5) Nearly every state uses a predictive instrument in at least one phase of criminal proceedings, as does the federal government. (6)

   The academic literature on risk assessment instruments is voluminous but tends to center on a small cluster of issues; namely, racial parity, predictive validity, and transparency. (7) That focus is too narrow. Several widely used risk assessment instruments incorporate attitude and associational factors by scoring subjects' abstract beliefs, peer associations, and family criminal history. (8) The Level of Service Inventory-Revised (LSI-R), for example, scores subjects on whether they have an "[u]nfavorable attitude toward convention," "[s]ome criminal acquaintances," and "[c]riminal family/spouse." (9) Such items raise significant constitutional, ethical, and empirical issues.

   Evidence is not necessarily admissible just because it is useful. Courts are prohibited from considering certain information when the value of punishing crime is outweighed by the protection and vindication of a defendant's constitutional rights. (10) Risk assessment instruments that try to quantify attitudes and associations can run afoul of constitutionally protected freedoms. In the sentencing context, certain speech and associational evidence is subject to an absolute bar under the First Amendment. (11) At all stages of criminal proceedings, evidence of family criminal history should be prohibited as a matter of Fourteenth Amendment equal protection. (12)

   Omitting attitude and associational factors from risk assessment is not just a matter of constitutionality but of pragmatism. The predictive utility of all attitude and associational factors has been overblown. (13) They are rife with measurement problems and bear only a nominal demonstrated correlation with recidivism. (14) These shortcomings bear on constitutional balancing tests for speech and family affiliation factors and counsel against the use of peer association factors. (15)

   The few legal scholars who have addressed attitude and associational factors in criminal risk assessment have neither questioned their empirical justification nor considered that they may violate the First Amendment. Dawindar Sidhu has challenged the use of speech and associational factors; however, his challenge relies on normative arguments and reference to Tocqueville, not the First Amendment. (16) Melissa Hamilton has briefly addressed the constitutionality of "procriminal attitudes" as a criminal risk factor, (17) but only through an equal protection lens. (18) J.C. Oleson touches on the issue of freedom of association in an article on constitutional issues in evidence-based sentencing, (19) but treats it primarily as a philosophical, rather than legal, issue. (20) He summarily assumes that a First Amendment speech challenge could not stand: "Most courts would not uphold defendants' challenges to evidence-based sentencing based on free speech, double jeopardy, or trial by jury rights, but some courts would be sympathetic to equal protection claims." (21) Similarly, Sonja Starr challenges whether sentencing based on a defendant's "demographic, socioeconomic, and family characteristics can be constitutionally or normatively justified" but seems to presume that sentencing based on a defendant's "conduct, mental states, and attitudes" would be a justifiable alternative. (22)

   Prior scholarship has likely overlooked problems with attitude and associational factors because it has focused on what factors are assessed instead of how factors are assessed. Identifying the issues with attitude and associational factors requires thorough examination of instrument questionnaires and scoring guides, as well as attention to the factors' empirical underpinnings. There are devils in the details. This Article aims to root them out.

2. Article Overview

   This Article is intended to be useful and accessible to all stakeholders in criminal risk assessment, including scholars, legal professionals, policymakers, and instrument developers. Legal scholars are often unfamiliar with instruments' content and empirical limitations, while other stakeholders are likely unfamiliar with the legal precepts that render certain information inadmissible in court. The legal and empirical issues in criminal risk assessment are intertwined, though. All stakeholders need the foundational legal and empirical knowledge to ensure the legal system is just and evenhanded.

   Part I of this Article provides an overview of several risk assessment instruments currently used in the United States. Part II explains and critiques how risk assessment instruments quantify "criminogenic needs." Part III discusses freedom of speech and the ethical problems inherent to quantification of attitudes. Part IV addresses peer and family associational factors, delving into the constitutional quagmire that is freedom of intimate association, the reasons "criminal family" factors violate equal protection, the traces of eugenics that persist in risk assessment literature, and the ethical and statistical problems with efforts to tabulate criminal associates and family members. Finally, Part V briefly expands the scope outward to caution against implementing poorly designed and validated instruments just because risk assessment is trendy.

   1. Overview of Select Risk Assessment Instruments

   This Part will provide an overview of four risk assessment instruments used by numerous American states, based on information and resources made public by developers, non-profits, scholars, and reporters. The instruments discussed in this Article were selected because they include attitude and associational factors, are not offense-specific, and are employed at sentencing in at least one state. They represent a significant fraction of the vast alphabet soup of instruments the federal government, states, and private developers have produced for use in criminal proceedings. (23)

   Discussing the current content of risk assessment instruments requires a certain amount of guesswork and extrapolation. Criminal risk assessment instruments typically enjoy limited transparency despite their ubiquity, (24) and presentation of their results can vary from state to state or even county to county. (25) This Article presumes, based on publicly available presentence reports, briefs, and opinions, that the risk assessment report that goes to a judge includes, at most, the defendant's overall risk score, scores in each risk and need category, and a few details noted by the assessor. (26) The Article also presumes that judges do not see instruments' underlying questionnaires, interview guides, or scoring guides. (27)

1. Correctional Offender Management Profiling for Alternative Sanctions (COMPAS)

   COMPAS is an instrument developed in the mid-1990s by psychometrician Tim Brennan and corrections professional Dave Wells and licensed by the company equivant (formerly Northpointe) to many jurisdictions. (28) COMPAS is likely the best-known and most-scrutinized criminal risk assessment instrument currently in use because of a 2016 constitutional challenge in Wisconsin and a ProPublica investigation that alleged...