Everything Is Not Terminator Is China's Social Credit System the Future?

• Citation: Vol. 2 No. 6
• Publication year: 2019

John Frank Weaver^*

In 2014, the central government of the People's Republic of China announced a "social credit" system, which will monitor the behavior of the nation's population and rank each person based on his or her social credit, i.e., how well each person behaves as a citizen. The term social credit dates to 2002, when the central government was looking at establishing a system to measure the creditworthiness of its citizens, but ultimately wanted the system to be compatible with the social services system.¹ Forty-three municipalities are currently testing versions of the system, and the program is scheduled to be operational nationwide by 2020.² To track citizens, the municipalities' systems rely on huge amounts of personal data from a wide variety of sources, including social networks, smart phone apps, and video cameras already installed by the central government. Those cameras are part of "Skynet," the Chinese government's video surveillance system, the public purpose of which is to track criminal behavior, but which has more than 20 million cameras in public spaces across the country.³ The end goal is to rely on sophisticated artificial intelligence ("AI") to review all of this data and add and deduct points from citizens based on how well they engage in lawful behavior.⁴

The Future with AI

Not surprisingly, this has prompted a great deal of criticism.⁵ However, I can't help but wonder if this is the natural development of the future with AI.

One theme that I have returned to frequently when writing and speaking about AI is what laws and regulations should do in response to fundamental changes in society introduced by technological changes.⁶ In particular, the century of legal changes in the wake of the American Industrial Revolution is a useful (albeit slow) example of what government can do to spread widely the benefits of massive technological advancements.⁷ The introduction of the factory system moved unprecedented numbers of people from rural communities to urban communities and shifted their work from farming to manufacturing, from work hours based on the seasons and the sun to work hours based on the company clock. In response, Congress and state legislatures enacted legislation that instituted a minimum wage, limited how many hours a person could work in a week, mandated workplace and environmental standards, and prohibited children from working.⁸ Essentially, elected representatives recognized that the Industrial Revolution had changed America and used laws and regulations to shape the new country into a version that was better for their constituents.

[Page 445]

If AI is going to introduce as much change as many experts predict, as much as the Industrial Revolution,⁹ then we should wonder what that changed world will look like. In the same way that it would have been hard but not impossible for leaders at the beginning of the Industrial Revolution to predict how the world was going to change, it is difficult but not impossible for us to predict how AI is going to change the world. Systems like China's social credit system might be what it looks like.

That should not be too surprising. There are other versions of it in place already. In the United States, credit-rating agencies score and rank individuals based on their creditworthiness, a system that the Chinese central government relied on when it created its social credit system.¹⁰ In the United Kingdom, there are systems in place to score individuals using data from a variety of sources, like credit score, phone usage, and rent payments; based on those scores, individuals are ranked by preference for job applications, access to social services, etc.¹¹ Similarly, in Germany there are systems in place to track data and rank individuals based on creditworthiness and healthy lifestyle.¹² China's proposal is therefore not unique because of the theory it relies on, but because of its scope.¹³ That suggests that the world we live in is already marching toward national governments accepting that AI systems should analyze the vast amounts of personal data available to them and score citizens based on that analysis.

[Page 446]

The Future of AI Does Not Have to be a Bad One

If that is likely the future, does it have to be a bad one? Read any of the articles in the footnotes of this column and you are likely to see the terms "dystopian," "chilling," and "Black Mirror" tossed around. That threat is real, and the possibility of a social credit system arbitrarily hurting already vulnerable populations is strong. It is all too easy to find stories of people in China who have been barred from jobs or schools,¹⁴ prohibited from purchasing certain train tickets,¹⁵ and banned from vacationing¹⁶ due to low social credit scores with no due process and for seemingly arbitrary or capricious reasons. However, I do not believe that is the inevitable result for two reasons.

First, the goal of the Chinese system and copycats in other nations is to reward good behavior and promote good citizenship, essentially through gamification.¹⁷ By assigning points to certain kinds of behavior, social credit AI systems...