How to Make the Perfect Citizen? Lessons from China's Social Credit System.

• Author: Orgad, Liav

TABLE OF CONTENTS I. INTRODUCTION 1088 II. SOCIAL CREDIT SYSTEMS IN CHINA 1091 III. CHINA AND LIBERAL DEMOCRACIES 1101 IV. CIVIC VIRTUE 1107 A. Instrumentalization and Standardization 1109 B. Normative Critique 1112 C. Cybernetic Citizenship 1117 V. CONCLUSION 1120 I. INTRODUCTION

Imagine a world where your daily activities are constantly watched and recorded: what you buy, whether you volunteer in the community, how often you visit your parents, who your close friends are, where you travel, and when you pay your bills. The aggregated data are collected from dozens of public and private agencies and then processed and assessed by a central bureau for developing a ranking for you and your fellow citizens. A high ranking is rewarded while a low ranking may be sanctioned. The ranking is publicly known, so that people can check it before they contact you, date you, or do business with you. And although you are aware that you are constantly being rated, the data sources, criteria used, and ranking methods are largely unknown.

In 2014, the Communist Party of China introduced a plan to construct a "Social Credit System" ([phrase omitted]). (1) It intends to use technological innovation to establish a unified system that rates citizens for improving social order and public trust. Although one cannot speak yet of a Social Credit System, China aims to create a comprehensive ecosystem by the end of 2021, where all citizens (2) are rated based on a national credit database. The system should cover most, if not all, aspects of civic life: commerce, finance, taxation, employment, education, transportation, housing, scientific research, and even sports--almost everything that a person leaves a footprint of, physical or digital. It promises to reward "model individuals keeping trust" and impose sanctions for a breach of trust by "blacklist systems and market withdrawal mechanisms." (3) In order to keep trust, a person should obey the law and follow "professional ethics and behavioral norms." The overall idea is to forge public policy in which "keeping trust is glorious and breaking trust is disgraceful." (4)

The Chinese project has become infamous and has been portrayed by Western media as the world's first "digital dictatorship," a "form of high-tech Stalinism," and a means of Orwellian control. (5) Former U.S. Vice President Mike Pence had declared that "China's rulers aim to implement an Orwellian system premised on controlling virtually every facet of human life--the so-called 'Social Credit Score."' (6) Western reports on the Social Credit System present the ultimate dystopian nightmare, a world in which individuals live in "Bentham's panopticon." (7) There is a sense of anxiety of a future regime ruled by a "Metric Society," a "Black Box Society" and "Surveillance Capitalism." (8) In this background, the Social Credit System is a Pandora's box where everyone expresses their fears. Science fiction series, such as Black Mirror's "Nosedive" episode (2016), accelerate the fears of a society where people become a commodity in the "marketplace of scores." (9) But even though the Social Credit System is a Chinese project, it also reflects a concern in liberal democracies; it is a striking reminder of how the world is rapidly changing. (10)

The Article analyzes social credit systems from the perspective of citizenship in liberal democracies. Thus far, scholarly discussions have largely ignored the impact of emerging technologies on the institution of citizenship. This neglect is remarkable given the increasing speed at which emerging technologies are entangled in the development of new forms of citizenship governance. To a large extent, the intellectual grounds of existing citizenship regimes had been developed long before the rise of automated systems and, therefore, have not engaged with what can be termed "cybernetic citizenship." Against this backdrop, the Article presents a comparison of how new technologies redefine the essence of citizenship in both China and liberal democracies. China's Social Credit System is a unique case as it represents one of the most ambitious attempts in history to use sociotechnical means to produce "perfect citizens." It demonstrates not only how new technologies transform citizenship values and institutions but also indicates future directions of governance. These implement fundamentally different conceptions of freedom and undermine one of the most significant achievements of the Enlightenment--the Kantian-rooted idea that human beings should be treated as an end in themselves, and not merely as a means to achieve public goods.

The Article proceeds as follows. Part II is empirical: it describes social credit systems in China at different levels--national, local, private--and identifies their data sources, criteria used, rating methods, and sanctions and rewards attached to them. Part III is comparative: it analyzes similarities and differences between social credit systems in China and scoring and rating systems in liberal societies. It suggests three points of divergence: scope (is the system all-encompassing or limited in terms of topics and applications to one field?), authority (is the system centralised or decentralised, private or public?), and regime type (is the system part of a democratic regime?). Part IV is normative: it argues that China's Social Credit Systems can be seen as a form of cybernetic citizenship governance that changes the essence of citizenship and the political role of the state. It examines this development from the perspective of political philosophy, discussing how social credit systems impact the concept of civic virtue. Overall, the Article invites the readers to reflect upon the challenges and opportunities brought about by rapidly developing systems of sociotechnical citizenship governance. This topic is likely to be on the agenda in the years to come, particularly in a post-COVID-19 world in which citizens may tolerate more surveillance means, including sensing technologies and self-tracking apps to monitor their lives.

2. SOCIAL CREDIT SYSTEMS IN CHINA

   There is no commonly accepted definition of a social credit system. At a minimum, it is a form of governance that systematically collects information on societal actors (input), usually by using surveillance techniques, and processes the data to rate those actors according to some categories (throughput), which, based on a system of carrots and sticks, aims to incentivise a certain type of behavior (output). Each actor is watched and recorded, assessed and rated, and either rewarded or sanctioned. It is an infrastructure mechanism for data collection and aggregation on a wide range of topics (shared between government departments and private organisations), data assessment and analysis whose outcome is some sort of rating, and data leveraged to encourage or deter some type of behavior by using rewards and sanctions. (11) A social credit system is not necessarily linked to technological progress, yet recent attempts to implement it present a new generation of systems that aim to develop a scientific understanding of society through the use of emerging technologies.

   In recent years, social credit systems are mainly discussed with reference to China. Despite the reports on a Social Credit System, a unified national system does not exist (yet) in China. And still, some forms of social credit systems, which are developed and implemented in China, provide adequate information to understand the systems' goals, functions, and outcomes. Overall, three types of systems are in place. First, there is the national system of redlists and blacklists curated by the government. If citizens are blacklisted, they can be limited in the type of services and goods they can access. Second, over a dozen pilot projects of the Social Credit System exist at the sub-national levels--all are diverse in terms of design and implementation. And third, there are private (12) and commercial initiatives, such as the Zhima (or Sesame) Credit.

   At the national level, the social credit initiative revolves around notions of redlists and blacklists, which are lists of "trustworthy" citizens, who should be rewarded, and "untrustworthy" citizens, who should be sanctioned. The first national blacklist was created by the Supreme People's Court in 2013; (13) it tracks down "dishonest persons" who are obliged and capable of complying with court orders and administrative decisions yet fail to do so. (14) Blacklisted people appear with their names on an official website of the Supreme People's Court. (15) Other blacklists have been created since 2013, either thematic, for a specific field (tax, transport, customs, etc.), or geographic, for a confined area (province, city, etc.). Grounds for being on a blacklist include failure to comply with a court order, tax evasion, scam activities, or misbehavior in public transportation. Grounds for being on a redlist include a good financial credit and payment record, social charity, blood donation, "good citizen" award, and environmental protection. (16) The result is dozens of national and regional lists, whose aggregated data is kept in a centralized database, the National Credit Information Sharing Platform (NCISP). The existence of "memoranda of cooperation" allows government agencies and the private sector to share data. (17) The data is organized by different categories, which, together, include hundreds of data points on each individual. It contains financial information, employment information, and information on legal conformity and violations, such as traffic violations, insurance fraud, and violations of a professional code of ethics. Legal limitations on data collection exist; for instance, collecting genetic or religious data is prohibited.

   The categorization into black and red lists--at the national level, people are not strictly speaking scored--has serious consequences. Blacklisted...