Machine learning, automated suspicion algorithms, and the Fourth Amendment.

• Author: Rich, Michael L.
• Position: Abstract through III. The Insufficiency of an ASA's Prediction, p. 871-901

At the conceptual intersection of machine learning and government data collection lie Automated Suspicion Algorithms, or ASAs, which are created by applying machine learning methods to collections of government data with the purpose of identifying individuals likely to be engaged in criminal activity. The novel promise of ASAs is that they can identify data-supported correlations between innocent conduct and criminal activity and help police prevent crime. ASAs present a novel doctrinal challenge as well, as they intrude on a step of the Fourth Amendment's individualized suspicion analysis, previously the sole province of human actors: the determination of when reasonable suspicion or probable cause can be inferred from established facts. This Article analyzes ASAs under existing Fourth Amendment doctrine for the benefit of courts that will soon be asked to deal with ASAs. In the process, this Article reveals the inadequacies of existing doctrine for handling these new technologies and proposes extrajudicial means for ensuring that ASAs are accurate and effective.

INTRODUCTION I. MACHINE LEARNING AND ASAs II. INDIVIDUALIZED SUSPICION, OLD ALGORITHMS, AND ASAs A. The Two-Step Individualized Suspicion Analysis B. Algorithms in the Individualized Suspicion Analysis: The Old and the New III. THE INSUFFICIENCY OF AN ASA's PREDICTION A. The Collective and Constructive Knowledge Doctrines B. Applying the Doctrines to ASAs IV. INCLUDING ASAS IN THE TOTALITY-OF-THE-CIRCUMSTANCES ANALYSIS A. Algorithms as Police Profiles B. Algorithms as Informants C. Algorithms as Drug-Sniffing Dogs 1. The Law of Drug Dogs 2. ASAs as Drug Dogs 3. Conclusion V. ASA ERRORS CONCLUSION INTRODUCTION

One day soon, a machine will identify likely criminal activity and, with the beep of an e-mail delivery, the buzz of an alarm, or the silent creation of a report, tell police where to find it. Already, a computer program analyzes massive quantities of securities trading data and notifies the Securities and Exchange Commission of investors who might be engaged in insider trading. (1) Computer systems connected to networks of video cameras alert police when bags are abandoned on subway platforms, (2) when people on a street corner interact multiple times in a short period, (3) or when a single individual visits multiple cars in a parking structure. (4) The federal government has field tested a device that screens individuals and predicts whether, based on physiological data, the individual intends to commit a terrorist act. (5) Researchers at Carnegie Mellon, funded by the Defense Advanced Research Projects Agency, are developing computer systems to index and analyze the text and images in online advertisements for sex services to identify likely sex traffickers and their victims. (6) While these current technologies generally follow a comprehensible logic--looking for facts that we understand to correlate with criminal conduct--technologies of the near future will analyze more data than a human being could and unearth connections that evade obvious logic.? In other words, soon a computer may spit out a person's name, address, and social security number along with the probability that the person is engaged in a certain criminal activity, with no further explanation. (8)

These emergent technologies arise from the intersection of two trends: the collection of massive troves of individualized data about people in the United States and the explosive growth of a field of computer science known as machine learning. (9) With respect to the former, these data come from a nearly unlimited variety of public and private sources, including video cameras, crime scene gunshot detectors, license plate readers, automatic tollbooth payment systems, and social media websites. (10) Government bodies from the municipal to the federal level are all involved in this "data vacuuming." (11) Moreover, private companies are increasingly making personal data available to governments including to law enforcement agencies. (12) With a mixture of resignation and pessimism, this Article takes the government's past and future collection of enormous quantities of personal data as a given and instead examines the government's use of those data. (13)

Meanwhile, researchers have made colossal strides in recent years in machine learning, "the systematic study of algorithms and systems that improve their knowledge or performance with experience." (14) Machine learning is particularly useful for revealing otherwise unrecognizable patterns in complex processes underlying observable phenomena. (15) Specifically, machine learning techniques help computer systems learn about an underlying process and its patterns by creating a useful mathematical approximation of how the process works. (16) This approximation can then be applied to new data to predict future occurrences of the same phenomena. (17) For instance, machine learning methods are used to examine patient records and create algorithms that can help doctors diagnose illnesses or provide prognoses. (18)

At least on a conceptual level, machine learning and crime fighting are a perfect match. The interaction of forces that cause people to commit crimes is incomprehensibly complex. Criminologists have sought for decades to use data to understand that interaction and identify the most likely criminal offenders. (19) Statistical models that aim to identify the criminally inclined based on quantifiable personal characteristics have become influential in the contexts of pretrial release, probation, and parole. (20) Similarly, police departments have recently begun to use statistical models to predict where in their jurisdictions certain crimes are likely to occur. (21) Machine learning provides a way to go one step further and use data to identify likely criminals among the general population without the need to disentangle the Gordian knot of causal forces.

This Article addresses technologies that apply machine learning techniques to the "data hoards" available to law enforcement in order to predict individual criminality. (22) Some of these technologies are already in use or are in advanced stages of development. (23) Nascent examples are even more numerous, including: using past offender and crime scene data to create more accurate profiles of unknown offenders, (24) leveraging behavioral data to identify individuals who are attempting to conceal their true--and potentially nefarious--intent, (25) and analyzing past corporate financial statements to create algorithms that can determine from the language used in a financial statement whether the company is likely engaged in fraud. (26)

This Article refers to programs like these--programs created through machine learning processes that seek to predict individual criminality--as Automated Suspicion Algorithms, or ASAs. ASAs share three defining characteristics as implied by the name. First, they are based on algorithms, which can be broadly defined as sequences of instructions to convert an input into an output. (27) In this case, ASAs convert data about an individual and her behavior into predictions of the likelihood that she is engaged in criminal conduct. (28) Second, ASAs assess individuals based on suspicion of criminal activity in that they engage in probabilistic predictions that rely on patterns detected in imperfect information. (29) Third, ASAs automate the process of identifying suspicious individuals from data: they comb through data for factors that correlate to criminal activity, assess the weight of each factor and how it relates to other factors, use the results to predict criminality from new data, and continuously improve their performance over time. (30) The automated creation of rules that predict criminality distinguishes ASAs from computer systems that might merely automate the application of a pre-existing police profile of criminality. (31)

Of course, from fingerprints to field testing kits to DNA matching, law enforcement has always tried to find ways to use the newest technologies. (32) As a result, attorneys, judges, and commentators are quite familiar with the role that technologies play in helping police ascertain the basic facts about a crime: the who, what, when, where, and why. A field test for cocaine, for instance, tells police whether a certain substance is contraband. A DNA match confirms that a suspect was at a crime scene. But determining these historical facts is only the first step in deciding whether individualized suspicion exists sufficient to justify a search or seizure under the Fourth Amendment. (33)

Until now, the second step in determining the existence of individualized suspicion--deciding whether the historical facts give rise to probable cause or reasonable suspicion (34)--has remained the sole province of human actors. The Supreme Court has held that determinations about the existence of probable cause and reasonable suspicion ultimately depend on reason, (35) "common sense," (36) and police experience. (37) The Court has also made clear that individualized suspicion is ultimately about "probabilities," though in the next breath we learn that probabilities "are not technical." (38) The promise of ASAs is that they can answer the individualized suspicion question by providing data-derived probabilities of whether crime is afoot; the novel problem they present is how those statistical probabilities fit in the "practical, nontechnical conception" of individualized suspicion articulated by the Supreme Court. (39)

ASAs are coming, (40) and courts will soon be asked to consider how their output should factor into the individualized suspicion analysis. (41) The initial goal of this Article is to provide courts with a framework for that analysis. (42) Yet setting out this framework teaches broader lessons about how emergent technologies interact with the Fourth Amendment. First, we learn that ASAs push the limits of the Court's...