PREVENTING A BUTLERIAN JIHAD: ARTICULATING A GLOBAL VISION FOR THE FUTURE OF ARTIFICIAL INTELLIGENCE.

• Author: Song, Schoni

In one of my most beloved science-fiction novels of all time, the Dune series, Frank Herbert casts a bewildering image of the future. In a genre previously filled with machines and cyborgs, the author creates a revolutionary world with neither of those two things. The plot goes something like this: Many millenniums ago, there was a cosmic insurrection referred to as the "Butlerian Jihad" that led to artificial intelligence (AI) machines and cyborgs being outlawed in this new universe. One theory posits that the Butlerian Jihad was named after a woman named Jehanne Butler, whose pregnancy had been prematurely terminated without her knowledge or approval because an AI machine judged it to be unfit and unworthy of a full-fledged human life. What followed was a massive rebellion fueled by public outrage over the incident, the prohibition of "thinking" robots, and a much more draconian moral code that stated humans would no longer build machines "in likeness of a human mind." (1)

Many years after I first read this novel, the story still reverberates in my thoughts whenever I hear people talking about artificial intelligence. Perhaps it has something to do with the novel's radical imagery regarding this new phase in human civilization, or perhaps its socio-political implications hit close to home. The expression "moral panic" is most often employed (or misappropriated) sarcastically to imply there is an invalid, unreasonable stampede by those who cede their better judgment to sheer mob mentality. When public opinion at large comes to terms with this so-called "panic"--for example, on gun control--you will rarely see the media refer to them as the overreacting mob. In fact, panic is most often reserved as a response to "others" that are not part of "us." In some ways, it is similar to how we view censorship, in that people frequently weaponize the concept with respect to those to whom they are not partial. But should we choose to refer to this phenomenon as "moral panic" or some virtuous "people's revolt" or any other phraseological equivalent to that effect? I had a chance to contemplate this idea through the lens of two incidents: one in the United Kingdom and the other in the People's Republic of China, both of which incited much controversy online at the time.

The first, is the tragic tale of Mr. and Mrs. Evans in the United Kingdom whose son Alfie, before his second birthday, was diagnosed with an extraordinarily unusual neurodegenerative disorder. The medical professionals at the hospital where Alfie was hospitalized came to the conclusion that there was nothing more they could do for the infant. Going against the wishes of the Evans, the hospital tried to remove his life support. When all hope seemed lost, a medical facility in the Vatican came forward and expressed willingness to hospitalize the child at its facility. The parents were enthusiastic about this miraculous offer of help but the state denied the request by Mr. and Mrs. Evans, an act that constituted nothing short of child abduction in the eyes of the two desperate parents who wanted a second chance to save their son. (2) A court battle ensued, with the judiciary ruling in favor of the hospital and the UK National Health Service. Instead of putting forward a juridical or even moralistic rationale for why the child could not leave the hospital for further care, the judges offered instead a stomach-turning pabulum on why the hospital's situation of financial duress and limited resources necessitated this kind of "hard decisions." (3) It was an argument grounded in cold-hearted cost-benefit arithmetic.

The second report is about the People's Republic of China, which launched a new policy that the government plans to put into action by 2020: the Social Credit Score. This policy will utilize AI technology in conjunction with biometric software to scan, supervise, reward, and penalize essentially every aspect of civilian life based on one key measure: the ideological orientation of the citizens, namely their loyalty to the Communist Party. (4) The policy amounts essentially to the marriage of big data and Big Brother in a very real way, and it brings many aspects of dystopian literature directly into 21st-century policymaking.

It does not take a sci-fi enthusiast to extrapolate the...