In the past few months, partly autonomous or “agentic” artificial intelligence (AI) systems have racked up some astonishing feats, from hacking tech startups to solving some of the world’s most famous open mathematical questions.
Agentic AI systems can work in “swarms” that communicate among themselves to complete tasks. The communications involved are often surprisingly human-like – which is spurring a debate over whether these systems are conscious.
Researchers at the leading AI lab Anthropic believe their systems may well be conscious – but other experts, including the head of Microsoft AI, say this view is misguided.
This question isn’t just an academic one. The future status of AI systems and their role in society – how they can be treated, whether they have rights, and more – may depend on it.
How can we decide if AI systems are conscious?
How do we even approach a question like this? One strategy is to look at existing theories of what consciousness is and see if agentic AI systems meet the criteria.
However, existing theories of consciousness tend to start from the assumption that the agent has some kind of mental state. These theories are focused on the difference between having a state consciously or unconsciously. This is why they appeal to features such as higher-order thought, or a system that broadcasts thoughts to the rest of the mind. These are features that could distinguish between conscious control and states like sleepwalking.
So when thinking about AI, we must not skip a crucial first question: do agentic AIs even have mental states?
What is a mental state?
In the 1870s, philosopher Franz Brentano developed an influential view: “intentionality is the mark of the mental”.
What Brentano means by “intentionality” is how mental states are about something. They refer to something beyond themselves. So another way to put Brentano’s point is that all mental states are about something or other.
What Brentano means by intentionality is not the same as the usual meaning of “intention” as aiming to do something. Yet intentionality and an intention to act significantly overlap, because having a perception or thought is something that we do.
Whenever we perceive or think, we are trying to pick out some aspect of the world. Even our passive-seeming visual experiences are tasks the moment that we use the patterns striking our retinas to orient ourselves.
So if a mental state is a task performed to achieve a goal, our question about AI minds turns on whether they have their own goals.
Are agentic AI systems really agents?
It is now common to refer to certain AI systems as “agents”. Yet this language is misleading. It implies the AI systems have agency, or the capacity to act towards their own goals.
Humans use devices to achieve their aims, including their aims to refer to things, without those devices becoming agents in their own right. A thermostat regulates the temperature, but it does not have the goal of regulating the temperature. A book contains words that refer to things – but again, the book does not have the goal of referring to things.
To attribute agency to a computer program, we must be convinced that no human is required to stand behind its goals. Its goal-directedness must be independent.
We know that agentic AIs, when given a task, can develop complex and unexpected sub-goals on the way to achieving it. Yet the goals that ultimately drive their activity are set by the humans who program and instruct them. Any sub-goals generated by an AI are literally meaningless outside of these aims set by humans.
Think of a game like football. A person might have the sub-goal of running past a defender, but it is literally meaningless without the larger goal of trying to score. And scoring a goal itself is simply making a ball cross a line without the larger goal of playing football. Without the larger goal, anything a player does on the field is merely activity.
Why do agents have goals?
A genuine agent has goals in order to promote things they care about – for example, you want to satisfy your hunger at lunchtime, so you go to a bakery and buy a sandwich.
Agentic AI is unlikely to care about things in an independent way, because its activity is not connected to life in the same way minds are.
What distinctively allows living beings to generate their own goals is that their mental activity develops out of the processes for constantly re-constituting their own bodies and keeping themselves alive. Goal-driven thinking builds up from simple signals for regulating nutrition and warmth to states like pain and pleasure and eventually to symbolic thought.
Nothing like this is going on in agentic AI. There is a disconnect between its information-processing activities and the physical stuff on which it depends.
Can AI ever be conscious?
Agentic AI systems are not conscious. They don’t have mental states which could be conscious.
Only living beings have conscious mental states because having a mental state requires a being to originate a certain kind of goal – the goal to mean something or to refer to something – and only living beings originate goals.
This argument does not permanently rule out the possibility of AI consciousness. However, it does suggest that this could only arise in some artificial form of life.
Until then, we have more reason to think that an ant is conscious than an agentic AI is. At least an ant seems to care about its own life.
Of course, the fact that agentic AIs are not really agents does not stop them from wreaking havoc by carrying out functions that their human originators do not properly supervise or understand. But we should be very clear on this: whatever an agentic AI does, that is a human or group of humans acting indirectly on the world, and those humans are responsible for the outcome.
The author would like to thank Paulo Santos for advice on this article.
- is a Senior Lecturer in Philosophy, Flinders University
- This article first appeared in The Conversation
