DECEPTION BY DESIGN.

• Author: Willis, Lauren E.

Table of Contents I. INTRODUCTION 116 II. PROGRAMMED TO DECEIVE 121 A. Modern Marketing and Sales Design 121 1. Ubiquitous Data Collection 123 2. Connected Digital Interfaces 126 3. Machine Learning and Experimentation 127 4. Creative Artificial Intelligence 130 B. The Digital Environment Facilitates Deception 132 1. Benefitting from Consumers' Illusion that They Control Digital Interfaces 132 2. Exploiting Consumers' Online Habits 134 3. Targeting and Eliciting Vulnerability 142 C. Unchecked by Law, the Digital Design Process Will Produce Deception 147 III. DIGITAL DECEPTION IMMUNITY 151 A. Deceptive Business Conduct as a Legal Construct 153 B. Methods of Proving Deceptive Business Conduct 154 C. Barriers to Demonstrating Digital Deception 156 1. The Irrelevance of the Reasonable Person 156 2. The Absence of Intent 157 3. The Contraction of Customer Behavior Evidence 159 4. The Invalidity of Results from Current Methods of Proof 160 a. Lack of Population Validity 160 b. Lack of Ecological Validity 163 5. The Impracticability of Analyzing a Business's Digital Conduct 165 IV. FUTURE-PROOFING DECEPTIVE TRADE PRACTICES LAW 169 A. Adopting a Presumption of Business Causation 171 B. Updating Unfairness Doctrine 176 C. Reprogramming the Algorithms 181 1. Adding Constraints to the Optimization Function 181 2. Micro-targeting for Good 184 3. Limiting the Scope of Machine Control 186 V. CONCLUSION: FAIR MARKETING BY DESIGN 187 I. INTRODUCTION

Big data, ubiquitous tracking, and machine learning and other types of artificial intelligence increasingly shape business interactions with consumers. Through algorithms, businesses employ these tools to design advertising, sales portals, return and cancellation processes, pricing, and even products and services themselves. Ultimately, these algorithms are programmed to optimize profit. At the same time, digital interfaces can exploit features of the online environment to manipulate and deceive, a phenomenon so common that the term "dark patterns" has been coined for it. (1) Although dark patterns can be intentionally programmed, today's machine learning systems can teach themselves to deceive people even when humans have not designed them to do so. One of this Article's insights is that when deception of consumers is profitable, business communications and conduct designed by algorithms optimized only for profit will inevitably engage in deception. (2)

The law prohibits business representations, omissions, or practices that deceive or are likely to deceive consumers acting reasonably under the circumstances. (3) Prohibitions on deception are under-enforced for many of the same reasons consumer law generally is under-enforced: enforcement agencies lack sufficient resources and impose penalties too small to deter; (4) a single competitor rarely has sufficient incentive to sue; (5) and consumers under-report deception (6) and are blocked from filing suit by contractual fine print. (7)

But the technology that designs business interactions with consumers today also poses a new and heretofore unrecognized problem--it threatens to immunize deception of consumers from legal prohibitions on deceptive business practices. Technology today allows businesses to produce multitudes of unique permutations of online advertisements, websites, and software applications ("apps"). Each permutation can be tailored for and delivered to particular consumers in particular contexts at particular times. The resulting deluge of increasingly algorithmically-designed, micro-targeted, and ever-changing digital communications and conduct effectively renders the leading methods of proving misleading or deceptive business practices obsolete. In sum, not only is deception inevitable online, it also evades the legal apparatus intended to enjoin, punish, and deter it. Identifying and analyzing this issue are the primary contributions of this Article.

The first barrier to demonstrating digital deception today is that the machines that design online business materials lack intent. Although intent to mislead is not an element of federal or most state deceptive practices claims, testimony or business records showing an intent to mislead are frequently introduced as powerful evidence of such practices. Machines, however, lack intent, and a business today can design deceptive digital interactions without intending to mislead. That scienter can be difficult to prove, even where it exists, has long been recognized by the law and was one motivation for the omission of an intent requirement from most statutes prohibiting deceptive business practices. (8) But recognition of a true absence of any human deceptive intent on the part of the business engaging in digital deceptive practices is a new finding of this Article.

Another set of barriers to demonstrating digital deception flow from the micro-targeted nature of the design and delivery of digital business materials. (9) Most methods for proving consumer deception focus on specifically-identified marketing and sales materials. These methods include reliance on: factfinder or expert application of the "reasonable consumer" standard to specific materials; expert analysis of consumer test subject responses to those materials; and today's evidentiary gold standard, randomized controlled experiments showing the proportion of consumer subjects deceived by those materials. However, digital materials are designed not for the reasonable person but for ever-narrowing and increasingly unintuitive segments. Consumer testing and experiments flounder because subjects cannot be identified who match, in pertinent respects, the consumers to whom specific digital materials were directed. Successful micro-targeting entails reaching consumers in the contexts and at the moments when they are most likely to respond in the manner desired by the business. The relevant aspects of these contexts and moments cannot be recreated when factfinders, experts, or consumer test subjects examine the materials.

Finally, the volume of unique designs of digital business materials poses a practical obstacle to demonstrating that a business's digital practices are deceptive. Not all of the thousands of micro-targeted versions of a business's advertisements, websites, or apps can be analyzed for litigation purposes. Analyzing or testing even a statistically representative sample would exceed practicable limits.

How can the legal system restore the enforceability of prohibitions on deception of consumers in the digital age? Elsewhere, I have proposed a new consumer law paradigm that would address deceptive business practices along with other challenges to consumer protection and fair competition. (10) This Article's goal is narrower: to suggest how judicial treatment of unfair and deceptive business practices claims under existing law can and should adapt to address the evidentiary barriers to enforcing deception prohibitions in the digital age.

Without new legislation, there are at least two routes for future-proofing the law. The first route is judicial adoption of the presumption that when consumers have false beliefs about facts material to a transaction in which they have engaged--including a false belief that they have not engaged in the transaction at all--the likely source of those false beliefs is the business that will benefit. The business could rebut the presumption with evidence that an independent source was responsible for its customers' false beliefs.

But does the source of the false beliefs matter? Sales based on false consumer beliefs, even where the seller did not create consumers' confusion, undermine the law's consumer protection and fair competition goals. The second route to future-proofing the law against digital consumer deception is, therefore, for the law to recognize that no matter how consumers are deceived, exploiting consumer confusion for profit is inherently an unfair practice.

Both a presumption that the business that benefitted caused the false beliefs and a recognition that a transaction based on false consumer beliefs is unfair regardless of the source of those beliefs would move the focus from the business and its myriad of contacts with consumers to consumers themselves. (11) It would eliminate the need to pinpoint which particular representations, omissions, or practices in particular micro-moments deceived a business's customers. Instead, enforcement agencies, competing businesses, and private plaintiffs would be required to demonstrate that a business's customers transacted with the business under false beliefs about facts material to the transaction. Businesses would then have the incentive to program their algorithms and design their digital materials to produce accurate consumer beliefs and to optimize profit within that constraint. That is, the law would compel businesses to engage in fair marketing by design.

This Article is part of a larger conversation about the digital manipulation of consumers and the capacity of existing law to regulate the unintended consequences of algorithms and artificial intelligence, such as perpetuating race and gender discrimination and propagating online extremism. Many insights here are applicable to those broader topics. As the design of digital materials becomes more automated, these materials will not only inevitably deceive, but they will also inevitably manipulate consumers in other unfair ways, for both commercial and political advantage. (12)

Limiting the analysis here to deception rather than analyzing all dark patterns sets aside thorny questions about when marketing crosses from fair persuasion to unfair or abusive manipulation. (13) This is not to say that non-deceptive manipulation is not a problem, but rather that we lack societal consensus on where to draw the line. (14) A focus on deception is also warranted because it is one of the most commonly-pleaded claims in consumer protection cases. (15) In part...