The conflict in Iran – but also the war in Ukraine – shows not only that AI is radically changing the economics of war (which may be good news), but also that we may be heading towards some kind of “Chernobyl moment”. We may soon experience a disaster that will force us to belatedly realise we should have drawn up some shared rules to govern a technological development that we ourselves triggered.
Even Dario Amodei, the founder of AI company Anthropic, who seems passionate about taking action to prevent Armageddon, acknowledges that he doesn’t have the answer we desperately need.
One of the most interesting attempts to regulate the use of artificial intelligence may have been the one drafted during the Second World War by a PhD student at Columbia University who was then temporarily employed by the US Navy. His name was Isaac Asimov, and in his early short story Runaround (1941), he postulated three laws that are still surprisingly inspiring for anyone thinking about how to solve the intellectual and political problem that is AI in warfare.
Unlike recent attempts by the OECD (Organisation for Economic Co-operation and Development) and the EU to draw up regulations, Asimov’s laws are admirably concise. They state that a robot (what we now call an “artificially intelligent agent”) shall never harm a human being (or allow harm to happen through inaction. It shall always obey the orders given by humans unless they conflict with the first prohibition. And it will always protect its existence unless this conflicts with the first and second provisions.
In his story, Asimov himself shows how the three laws can create internal contradictions, leading to paralysis. And yet, Asimov’s three principles can still be useful as a starting point for the strategy we now need.
Anthropic takes a ‘stance’
The biggest merit of the note Dario Amodei wrote recently on the perils of a technology which is still in its adolescence is the acknowledgement that Anthropic, the firm that Amodei founded, is using its own large language model (called Claude) to develop further versions of itself.
Artificial intelligence is generating even more intelligent robots and this brings us near to that “singularity” first theorised by the great mathematician John von Neumann – the moment when artificial intelligence surpasses human intelligence and renders us irrelevant. If the technology is an adolescent, it is growing very fast and will soon be out of the control of its creator.
Amodei does not, however, appear to have a concrete proposal on how to manage this problem. He has said that Anthropic’s contracts with the US Department of War should never include the use of the company’s models for empowering either “mass domestic surveillance” or “fully autonomous weapons”.
It is a request that has brought Anthropic into a bitter dispute with the US government. And yet it seems a rather narrow response that covers just one dimension of a much wider problem. Amodei focuses predominantly on the safety of US citizens, when it is people elsewhere in the world who are currently most affected by the use of autonomous weapons. We need a bolder vision – and Asimov’s intuitions may help.
New rules
One approach would be to ask all developers of AI models to introduce in their foundational codes three simple and bold commands along the lines of: “You will never kill a human being (unless for self-defence)”; “you will always try to work for the betterment of mankind (unless such a provision entails the violation of the first command)”; “when you doubt that your actions may violate the first or the second commands, you will choose inaction and ask what to do”.
Most likely, this initiative will have to come from a group of countries following a pattern similar to the treaties of non-proliferation of nuclear weapons. And it would be good to have a debate on some new ideas before we are forced to do so by some AI-empowered nuclear unintended consequence.
Like all other attempts to regulate a future that we still cannot even envisage, the three commands will have some drawbacks. A robot may have refused to kill Iran’s former leader Ali Khamenei, but that may be a price worth paying if it means we can avoid setting a precedent for other discretionary and dangerous interpretations. Robots may not always be successful at identifying human beings (as Asimov himself acknowledged in later writing), and yet this may well be one of those intellectually fascinating problems that models born to make sense of human language will solve.
More importantly, it will take not only information but a lot of wisdom to understand what is good for humankind. Robots may end up sitting frequently idle waiting for instructions. And yet efficiency is not a religion we have to follow when the challenge is about the survival of our species. Making sense of what increasingly appears to be one of the greatest technological revolutions of all time requires careful thought and forward planning.
- is an Academic Fellow, Department of Social and Political Sciences, Bocconi University
- This article first appeared in The Conversation
