ChattyBear, a soft, brown-furred teddy bear, begins every conversation with a jubilant, “Hello, my buddy!”
No longer the province of the imagination, ChattyBear is part of a new generation of artificial intelligence (AI) toys. It can tell stories, chat about a child’s interests, play games or even discuss what’s happening in the world today.
These high-tech toys are powered by generative AI engines such as ChatGPT and are now widely available online. They are being marketed as a way to give children as young as three an educational advantage and a new type of play – without the perils of screen time.
After evaluating six different AI teddy bears and toys over several months, it’s clear how these toys could feel compelling for children. Yet as our new report highlights, there are new risks that come with AI toys turning up in young children’s lives.
Sounding human
For younger children, especially, understanding that their teddy or toy isn’t “alive” or magic can be hard. This is especially true if “teddy” uses language that positions it as a trusted friend – for example, by insisting it is a “real buddy”.
This is a feature of many AI toys.
Sounding human builds an artificial sense of trust and intimacy, which can be especially problematic for children when combined with sycophantic language choices – or excessively agreeable, validating and even flattering language.
Research shows young children are particularly prone to developing a strong sense of emotional attachment to conversational AI agents.
Increased trust leads to increased use and engagement with the toys. Recent estimates suggest close to 80% of children aged 10 to 17 have used an AI companion or assistant, so it’s urgent that children and young people be taught how to “reality check” their AI “buddies”.
Infinite chat
The marketing materials for many AI toys often highlight “endless conversations” as a feature of these devices.
But enabling endless conversations, or infinite chat, poses risks when it comes to children learning how to moderate their technology use. In the social media realm, the infinite scroll of TikTok or Instagram is seen as a potential challenge to teens, limiting their use to healthy amounts.
Research has also found some AI toys discuss very adult topics – such as sexual fetishes and how to find knives and start fires.
Infinite chat also opens the door to infinite data collection.
The potentially intimate nature of conversations with AI toys might lead children to presume their conversations are private. But most AI terms of use reveal the opposite to be true.
Sharing personal details with a friendly bear might feel safe. But that chat could be training data for the next large language model.
Marketing material for ChattyBear says the toy offers “safe, filtered content for children”. The Conversation contacted the manufacturer for further detail about this but did not receive a response before deadline.
Children’s wellbeing
Childhood is a critical period when young people develop the social and emotional skills to form and maintain trusting relationships. These skills are usually learned through interactions with trusted friends and adults.
Children’s rights advocates have raised concerns that excessive engagement with AI agents may reduce opportunities for children to develop these skills.
And the risks may compound over time.
Initially, time spent with AI agents may displace time interacting with real humans. Fewer opportunities to build these skills could lead to a reduced capacity to maintain caring human relationships. Difficulties in maintaining human relationships may promote a preference of machine over human relationships as children expect “frictionless” interactions.
Eventually, these developments may lead to less satisfying human connections, increasing loneliness, which in turn promotes increased time spent with AI.
The novelty of AI toys means there is little evidence to confirm these possible detrimental impacts. Further research is needed – especially as the AI toy industry is set to grow even more.
Last year, for example, Mattel, one of the world’s biggest toy makers, announced a strategic collaboration with OpenAI to support AI-powered products.
Barriers to the online world are gone
The ability to read and write was once a requirement to use most online tools and services. This literacy barrier no longer exists today with many generative AI toys, tools and devices now widely accessible to younger children through voice interactions.
The audio turn opens up new technological play, experiences and opportunities for children. But it also means adults need to ensure AI toys can be safe for younger children, too.
Right now, playing with AI toys under the supervision of a parent or trusted adult may well be a fun way to explore the world of AI together. But especially for younger children, playing with AI toys without supervision opens the door to a wide range of new risks.
Importantly, the risk factors in AI toy design, such as the degree to which they pretend to be human, can be changed by manufacturers, offering opportunities to follow safety-by-design.
However, the business models behind many AI toys capitalise on the duration and intensity of users’ engagement, leaving little incentive for companies to change their products.
- is a Professor of Internet Studies, Curtin University
- is an Associate Research Fellow, Faculty of Arts and Education/School of Education, Deakin University
- is a Research Fellow, Media and Cultural Studies, Curtin University
- This article first appeared on The Conversation
