“Just as we always suspected”, writes musician and writer Anna Goldsworthy in her new Quarterly Essay on AI, “the god that may destroy us is the god of ourselves.” This sentence captures the core of the essay: a personal meditation on what AI reveals about humanity. She ponders not logic gates or quantum chips, but the technology’s moral, creative and even theological implications.
Goldsworthy skilfully moves between conversations with her AI-literate children and reflects on AI’s existential risks, education, labour, art, loneliness, companionship, corporate power and future. She asks whether humanity is wise enough to live with a technology that amplifies its weaknesses.
Goldsworthy treats AI, by which she largely means generative AI, as a mirror rather than a monster. Her most persuasive insight is that AI is not alien to us. It has learnt from the huge archive of human writings: the useful parts, but also the evasions, fantasies, cruelties and consolations.
Her theological framing works because the “god” in the title is not a deity arriving from elsewhere: it is something we have made, and it carries our fingerprints.
Technically, some of the timelines Goldsworthy discusses for artificial general intelligence – “a hypothetical AI system whose abilities match those of a human across virtually all cognitive domains” – and artificial superintelligence remain speculative. For example, when Goldsworthy discusses automation, the issue is not really whether machines are conscious, but whether humans may be “relieved of meaningful occupation”. She asks why we should automate reflexively, and whether we truly wish “to be rescued from all process”.
Human dependence and power
I find her concerns strongest when read less as technical claims about AI versus human consciousness and more as warnings about human dependence, power and meaning.
The social risks she describes do not require full artificial superintelligence. Labour disruption, educational dependency, synthetic intimacy, misinformation, concentration of power and automation of judgement are already pressing issues.
Goldsworthy’s concerns belong to the same moral moment as Pope Leo XIV’s recent warning of AI’s risk to humanity, Magnifica humanitas. Both frame AI around human dignity, truth, work, social justice and the need for technology to serve humanity, rather than being used for the concentration of power. They both question not only what AI can do, but which idea of being human will survive its rise.
Goldsworthy does not discuss AI from a distance. She brings it into the kitchen, the classroom, the family, the workplace and the inner life. Early in the essay, after she thanks her son for dinner by saying it “enhances my quality of life,” her 16-year-old replies, “You sound like an AI.” She quotes her children, who represent the generation already growing up inside the AI transition, throughout. At one point, her son tells her the hope is for AI to move “beyond probability to reason”. He adds: “once you establish certain axioms, everything else follows.”
Through her sons, Goldsworthy shows us AI is not arriving in some distant future. It is already changing how young people argue, learn, joke, imagine and worry. This intimate engagement with AI is one of the essay’s strengths.
Creativity, work and companionship
As a musician and a writer, Goldsworthy understands that art is not just about creating an artefact. It is born of labour, boredom, practice, learning, frustration, discipline and embodied knowledge. This is where I found her essay strongly compelling.
AI may generate text, music, images and designs at extraordinary speed, but speed is not the only value in creativity. Sometimes the slow and difficult process is the point. A pianist does not only want the sound of the piano; she wants the struggle, the touch, the memory of practice, the physical relationship with the instrument. A writer does not only want a finished paragraph. She may also need the thinking that happens while wrestling with it.
The essay raises valid concerns about work. Goldsworthy is right to question the assumption that every task that can be automated should be automated. She distinguishes between being freed from unnecessary busywork and being deprived of meaningful occupation.
As she puts it, “it is one thing to be delivered of busywork; it is another to be relieved of meaningful occupation.” I agree with this distinction. The danger, then is not just unemployment, but the quiet erosion of meaningful work, leaving humans to supervise, correct or consume what machines produce.
Her discussion of AI companionship is similarly thoughtful. She does not mock loneliness, nor does she dismiss the emotional attachments people may form with artificial companions. AI systems may offer comfort, attention and a sense of being heard, she acknowledges. But this also makes them powerful. A system that can simulate intimacy, remember preferences and respond with endless patience can become deeply persuasive.
The ethical concern is not whether the affection feels real to a human. It is who controls the system, what incentives shape it, and whether vulnerability becomes a business model.
Not every AI problem is unsolvable
As an AI expert, I would have liked a little more distinction between different kinds of AI systems and different levels of risk. Not every concern about AI belongs in the same category. Not every problem we hear about AI is unsolvable.
Hallucination, bias, copyright, labour displacement, emotional dependence, surveillance, autonomous weapons and existential risk are related – but they are not the same problem. They require different lenses. At times, though, the essay brings them close together under one large anxiety, without sufficiently distinguishing between them.
For example, in one passage, she moves from “nuclear hacking and bioterrorism” to AI-enabled blackmail, personalised manipulation, and what she calls “another order of mind control again”, before turning to Bostrom’s paperclip thought experiment.
The experiment imagines a scenario where a powerful AI engine is tasked to make paperclips. This seemingly trivial goal could end up with a catastrophe if the AI takes the task literally, to the extent that it resists shutdown and treats humans as obstacles – blocking it from achieving its goal. Goldsworthy uses this example to show the potency of AI’s power in pursuing a goal too literally.
The compression of her thought-jumping may be emotionally effective, but it can also blur the policy and technical distinctions that matter for governance.
Seriously unsettling
Goldsworthy is not writing a technical taxonomy of AI risk, though. She is asking a deeper, perhaps more difficult question: what happens to human beings when machines become fluent in the things we once thought only belonged to us? A chatbot can advise, flatter, comfort, imitate and calculate. That does not make it human.
That is why the essay succeeds. It is not a perfect technical map of AI, but it does not need to be. It is a serious, elegant and unsettling essay about humanity at the edge of a technology it does not fully understand and may not fully control.
Its strongest warning is not that AI is inhuman. It is that AI may reflect us too closely. The question Goldsworthy leaves us with is not only whether AI will become powerful, but whether we will become wise enough to live with what we have made.
- is a Professor in Computational Intelligence, Australian Catholic University
- This article first appeared in The Conversation
