Controlling Artificial Intelligence

18th July 2018

News, Student news, Alumni news

“We are several breakthroughs away from having superhuman AI, but when they come things could change very fast” warns Wadham alumnus, Professor Stuart Russell.

  • Professor Stuart Russell

    Professor Stuart Russell

Most people think that we will invent superintelligent AI and then figure out what to do with it.

Professor Stuart Russell

Professor of Computer Science at the University of California, Berkeley, Russell has devoted his career to the study of Artificial Intelligence, including such topics as the interaction of knowledge and machine learning, the unification of logic and probability, and metareasoning (reasoning about reasoning).

Currently on a sabbatical in Paris, Russell is working on a new book about human-compatible AI – meaning AI that is compatible with human existence. He describes the potential advent of superintelligent AI as the ‘biggest event in human history’.  It might allow us to address the problems and risks facing mankind and to usher in a golden age, he says, but only if we humans can maintain control over AI.

“Most people think that we will invent superintelligent AI and then figure out what to do with it. By and large the field has not spent time thinking about the control problem,” he says, despite warnings which started in the 1950s. 

Alan Turing predicted back in 1951 that machines would eventually take control. In 1965, I.J. Good, a statistician who had worked with Turing at Bletchley Park, described the ‘intelligence explosion’, where machines would rapidly and recursively redesign themselves to become vastly more intelligent than humans; Good expressed the hope that “the machine is docile enough to tell us how to keep it under control.”  After seeing a machine teach itself to play draughts far better than its creator, mathematician Norbert Wiener wrote a paper on the potential threat to humanity. He was perhaps the first to point to the specific nature of the problem: “We had better be quite sure that the purpose put into the machine is the purpose which we really desire.” Otherwise, explains Russell, you risk a scenario like that of King Midas: your wish is carried out, leading to an irreversible catastrophe. Furthermore, whatever objective you give a machine, it wants to remain in existence to carry it out, so it will defend itself and acquire the necessary resources to achieve its objective.

Despite these warnings, people within the field have carried on developing AI and it wasn’t until the early 2000s that questions started to be asked about control or safety.

The biggest worry would be if AI becomes a tool for national competition.

Professor Stuart Russell

Much of Russell’s work focuses on solving the problem of making machines that can help resolve the world’s problems but will defer to humans, asking permissions and allowing themselves to be switched off—no matter how intelligent they are. He believes we have the beginnings of a solution to the problem: “If the machine is uncertain of the true objective then we can prove that it will defer to humans. This is the basis of maintaining control.”

Russell is also working on the fourth edition of his seminal textbook Artificial Intelligence: A Modern Approach.  Written with Peter Norvig, the first edition was published in 1994 since when it has become the standard for anyone studying AI. This new edition will feature a section on the control problem and will rework much of the foundational material to emphasize the importance of uncertainty in objectives.

Will we be able to retain control over machines indefinitely? “Hard to say,” says Russell.  “The biggest worry would be if AI becomes a tool for national competition. To help prevent this, major AI groups in Western countries have agreed to share their work on the control problem and make everyone aware of significant steps towards human-level AI.”

But the fear of the deliberate misuse of AI remains. In a recent short film Slaughterbots, the horrifying reality of AI combined with weaponry is shockingly demonstrated when autonomous intelligent weapons are used with disturbing results. The film, featuring Russell’s comments at the end, has been viewed an estimated 70 million times.

“The concept of autonomous weapons as weapons of mass destruction was not getting through and we wanted to do something to raise awareness,” said Russell. Discussions at the UN regarding a treaty banning autonomous weapons are ongoing, but Russell points out that Britain, the US, and Russia are all opposed to a treaty.

Avoiding what Russell terms the ‘enfeeblement of human civilisation’ is also vital he says. “We pass on our civilisation through human minds and the danger of letting intelligent machines take on that job is that we become mere beneficiaries—it ceases to be a human civilisation.”

Russell worries about the detrimental effect of tribal attitudes towards AI – groups that are strongly for or against its development claiming only its advantages or disadvantages and thus preventing rational discussion of the problems and the finding of solutions. He cites Chernobyl and the self-destruction of the nuclear industry and the GMO (genetically modified organisms) debate as examples of this behaviour. Corporate self-interest also plays a damaging role he adds. “We have to have honest, sensible debate about how to proceed.” ­

“In the long run I am an optimist but sometimes it is hard to be optimistic. This is a fascinating time but we will need to make some big decisions about how the future will unfold,” he concludes.

Professor Russell was talking to Wadham's Head of Communications, Julia Banfield, while he was in Oxford to give the prestigious Strachey Lecture at the Sheldonian Theatre.

We will need to make some big decisions about how the future will unfold

Professor Stuart Russell


Professor Stuart Russell’s interest in machine intelligence was sparked when studying A-level computing at a local technical college while a pupil at St Paul’s School in London. After a gap year where he gained programming experience at IBM, Russell studied Physics at Wadham (1979-82) under Donald Edmonds and Geoff Brooker. After receiving his BA with first class honours he went to Stanford to do his PhD in Computer Science which he achieved in 1986. 

He then joined the faculty of the University of California at Berkeley, where he is Professor (and formerly Chair) of Electrical Engineering and Computer Sciences, holder of the Smith-Zadeh Chair in Engineering, and Director of the Center for Human-Compatible AI. He has served as an Adjunct Professor of Neurological Surgery at UC San Francisco and as Vice-Chair of the World Economic Forum's Council on AI and Robotics. He is a recipient of the Presidential Young Investigator Award of the National Science Foundation, the IJCAI Computers and Thought Award, the World Technology Award (Policy category), the Mitchell Prize of the American Statistical Association, and Outstanding Educator Awards from both ACM and AAAI. From 2012 to 2014 he held the Chaire Blaise Pascal in Paris. He is a Fellow of the American Association for Artificial Intelligence, the Association for Computing Machinery, and the American Association for the Advancement of Science. His book "Artificial Intelligence: A Modern Approach" (with Peter Norvig) is the standard text in AI; it has been translated into 13 languages and is used in over 1300 universities in 118 countries. His research covers a wide range of topics in artificial intelligence including machine learning, probabilistic reasoning, knowledge representation, planning, real-time decision making, multitarget tracking, computer vision, computational physiology, and philosophical foundations. He also works for the United Nations, developing a new global seismic monitoring system for the nuclear-test-ban treaty. His current concerns include the threat of autonomous weapons and the long-term future of artificial intelligence and its relation to humanity.

Related news