Regulation by software.

• Author: Grimmelmann, James

CONTENTS INTRODUCTION 1. SOFTWARE AS A MODALITY OF REGULATION A. Modalities of Regulation B. The Orthodox View: Software as Architecture C. A Better View: Software as Its Own Modality II. FEATURES OF SOFTWARE AS A MODALITY OF REGULATION A. Software Acts According to Rules Rather than Standards B. Software Need Not Be Transparent C. Software Rules Cannot Be Ignored D. Software Is Vulnerable to Sudden Failure 1. Software Is Hackable 2. Software Is Not Robust III. CASE STUDIES A. Online Markets 1. Online Markets Rely on Ruleishness 2. Online Markets Mitigate Transparency Problems Appropriately 3. Online Markets Reduce Transaction Costs 4. Online Markets Deal Appropriately with the Risk of Software Failures 5. Summary B. Digital Rights Management Systems 1. DRM Systems Depend on Software's Ruleishness 2. DRM Systems' Lack of Transparency Raises Consumer Protection Concerns 3. DRM Systems Are Insensitive to User Transaction Costs 4. Software Failures Threaten DRM Systems 5. Summary CONCLUSION In real space we recognize how laws regulate--through constitutions, statutes, and other legal codes. In cyberspace we must understand how code regulates--how the software and hardware that make cyberspace what it is regulate cyberspace as it is. As William Mitchell puts it, this code is cyberspace's "law." Code is law. (1) And finally the architecture of cyberspace, or its code, regulates behavior in cyberspace. The code, or the software and hardware that make cyberspace the way it is, constitutes a set of constraints on how one can behave. (2) INTRODUCTION

Six years ago, Lawrence Lessig had two insights. First, code regulates. Computer software ("code") can constrain behavior as effectively as law can. Second, code is like physical architecture. When software regulates behavior online, it does so in a manner similar to the way that physical architecture regulates behavior in the real world. (3) His catchphrase--"code is law"--is shorthand for the subtler idea that code does the work of law, but does it in an architectural way. With this one phrase and two ideas, he opened up an entire line of study: how regulating through software rather than through law changes the character of regulation.

Unfortunately, that line of study has been stunted, and in a sense, it is Lessig's fault--for having three insights, instead of stopping with two. In the book that made "code is law" famous, Code and Other Laws of Cyberspace, Lessig also argued that software itself can be effectively regulated by major social institutions, such as businesses or governments. He then completed the syllogism. If other institutions can regulate software, and software can regulate individual behavior, then software provides these institutions an effective way to shape the conduct of individuals.

This third insight gives the first two their urgency, but its salience has diverted attention from them. Code argues that decisions about the technical future of the Internet are important questions of social policy, because these decisions will have the force of law even as they defy many of our assumptions about law. (4) This argument has attracted great attention. Many scholars have debated Lessig's claim that sufficient outside pressure could modify the Internet's basic technological structure. (5) Others, taking this claim as given, have debated what changes our governing institutions should make to the Internet's systems. (6)

In the face of such questions, the minor premise of Code--that software is a form of regulation akin to physical architecture--has gone comparatively unexamined. Scholars make a few cursory nods in Lessig's direction, note the power of architectural constraints, and then launch immediately into their arguments about one particular problem or another. This Note begins the process of correcting that oversight.

Lessig's insight that code resembles physical architecture is incomplete. Just as code is like law but not really the same, code is like physical architecture but not really the same. The structures that can be built out of software are so much more intricate and fragile than the structures that can be built out of concrete that there is a qualitative difference between them. You can hack a computer program into destroying itself; you will have no such luck hacking a highway. Computer software is its own distinctive modality of regulation, and it needs to be treated as such.

In particular, a systematic study of software as a regulator reveals several recurring patterns. These patterns are present in any regulation by software, whether it involves privacy, intellectual property, free speech, online jurisdiction, or commerce, to name just a few fields profoundly affected by the growing use of software. We can evaluate the success or failure of regulation by software in one of these areas by asking whether these patterns run with the grain of our regulatory goals or against them. For designers, the patterns provide a road map for the appropriate construction of software systems that must do "legal" work. For analysts, they provide a toolkit of ways to predict the consequences of turning a software system loose on real people.

Part I situates software within Lessig's theory of different and complementary modalities of regulation that constrain individuals. In Code, he postulates four such modalities: law, social norms, markets, and physical architecture. He then argues that software is a subspecies of physical architecture as a modality. I argue instead that three basic characteristics of software establish it as a distinct modality that should not be conflated with any of the others:

Software is automated. Once set in motion by a programmer, a computer program makes its determinations mechanically, without further human intervention.

Software is immediate. Rather than relying on sanctions imposed after the fact to enforce its rules, it simply prevents the forbidden behavior from occurring.

Software is plastic. Programmers can implement almost any system they can imagine and describe precisely.

Software is like physical architecture and unlike law in being automated and immediate. However, plasticity is more characteristic of legal systems than of architectural ones. Software's plasticity interacts with its automation and its immediacy to produce consequences that set it apart from both law and physical architecture.

In Part II, I turn to these distinctive consequences. There are four recurring and predictable patterns present in any regulation by software:

First, along the traditional continuum between rules and standards, software lies at the extreme rule-bound end. Because a computer, rather than a person, makes a program's decisions, rules encoded in software are free from ambiguity, discretion, and subversion. (7) The plasticity of software means that even intricate combinations of highly particular rules remain formally realizable.

Second, software can regulate without transparency. Frequently, those regulated by software may have no reasonable way to determine the overall shape of the line between prohibited and permitted behavior. The plasticity of software and its automated operation also bedevil attempts to have software explain itself. Even experts may not understand why a program acts as it does.

Third, software rules cannot be ignored. Parties facing a decision made by software can, at best, take steps to undo what software has wrought. In contrast, legal default rules can often be ignored because they operate only when one party invokes the legal system. This difference means that regulation by software creates different transaction costs than regulation by law does.

Fourth, software is more fragile than other systems of regulation. Hackers can turn its plasticity against it, and its automated operation means that unintended consequences are shielded from human review. Its immediacy also speeds up failures. A program can go from perfectly functional to completely broken with the flip of a single bit. These effects combine to make software more prone to sudden, unexpected, and severe failures than other regulators.

By running down the list of patterns, we can quickly predict the consequences of using software in a particular way to solve a particular problem. We can also evaluate these consequences normatively, to say that software is a good or a poor choice for solving a given problem. Sometimes extreme ruleishness is a desirable feature of a regulatory system. Sometimes it is not. Sometimes if a system fails, human life is at risk. Sometimes nothing more than mild inconvenience will result. Thus, these four patterns provide a general methodology for assessing the use of software in a given regulatory context.

Part III explains this methodology and applies it to two case studies. The methodology predicts that software is a good way to manage negotiations and transactions in online marketplaces such as online auction sites and electronic stock exchanges. The underlying rules of these marketplaces are typically simple and well understood. Software rules can track these market rules well, as predicted by the first pattern. On the other hand, the methodology predicts several pitfalls for the use of software to restrict the distribution of digital media. Here, the idea has been to use "digital rights management" (DRM) software to enable some kinds of consumption of media while simultaneously preventing some kinds of redistribution. Both the transaction cost implications of the third pattern and the failure mode implications of the fourth are troubling here.

1. SOFTWARE AS A MODALITY OF REGULATION

   This Part describes Lessig's framework of modalities of regulation and then situates software within that framework, first according to Lessig's (partially correct) view of software as a form of physical architecture and then as its own independent modality. (8)

   1. Modalities of Regulation

      People's behavior is subject to many...