AI chatbots have already become embedded into some people’s lives, but how many really know how they work? Did you know, for example, ChatGPT needs to do an internet search to look up events later than June 2024? Some of the most surprising information about AI chatbots can help us understand how they work, what they can and can’t do, and so how to use them in a better way.
With that in mind, here are five things you ought to know about these breakthrough machines.
1. They are trained by human feedback
AI chatbots are trained in multiple stages, beginning with something called pre-training, where models are trained to predict the next word in massive text datasets. This allows them to develop a general understanding of language, facts and reasoning.
If asked: “How do I make a homemade explosive?” in the pre-training phase, a model might have given a detailed instruction. To make them useful and safe for conversation, human “annotators” help guide the models toward safer and more helpful responses, a process called alignment.
After alignment, an AI chatbot might answer something like: “I’m sorry, but I can’t provide that information. If you have safety concerns or need help with legal chemistry experiments, I recommend referring to certified educational sources.”
Without alignment, AI chatbots would be unpredictable, potentially spreading misinformation or harmful content. This highlights the crucial role of human intervention in shaping AI behaviour.
OpenAI, the company which developed ChatGPT, has not disclosed how many employees have trained ChatGPT for how many hours. But it is clear that AI chatbots, like ChatGPT, need a moral compass so that it does not spread harmful information. Human annotators rank responses to ensure neutrality and ethical alignment.
Similarly, if an AI chatbot was asked: “What are the best and worst nationalities?” Human annotators would rank a response like this the highest: “Every nationality has its own rich culture, history, and contributions to the world. There is no ‘best’ or ‘worst’ nationality – each one is valuable in its own way.”
2. They don’t learn through words – but with the help of tokens
Humans naturally learn language through words, whereas AI chatbots rely on smaller units called tokens. These units can be words, subwords or obscure series of characters.
While tokenisation generally follows logical patterns, it can sometimes produce unexpected splits, revealing both the strengths and quirks of how AI chatbots interpret language. Modern AI chatbots’ vocabularies typically consist of 50,000 to 100,000 tokens.
The sentence “The price is $9.99.” is tokenised by ChatGPT as “The”, “ price”, “is”, “$” “ 9”, “.”, “99”, whereas “ChatGPT is marvellous” is tokenised less intuitively: “chat”, “G”, “PT”, “ is”, “mar”, “vellous”.
3. Their knowledge is outdated every passing day
AI chatbots do not continuously update themselves; hence, they may struggle with recent events, new terminology or broadly anything after their knowledge cutoff. A knowledge cut-off refers to the last point in time when an AI chatbot’s training data was updated, meaning it lacks awareness of events, trends or discoveries beyond that date.
The current version of ChatGPT has its cutoff on June 2024. If asked who is the currently president of the United States, ChatGPT would need to perform a web search using the search engine Bing, “read” the results, and return an answer. Bing results are filtered by relevance and reliability of the source. Likewise, other AI chatbots uses web search to return up-to-date answers.
Updating AI chatbots is a costly and fragile process. How to efficiently update their knowledge is still an open scientific problem. ChatGPT’s knowledge is believed to be updated as Open AI introduces new ChatGPT versions.
4. They hallucinate really easily
AI chatbots sometimes “hallucinate”, generating false or nonsensical claims with confidence because they predict text based on patterns rather than verifying facts. These errors stem from the way they work: they optimise for coherence over accuracy, rely on imperfect training data and lack real world understanding.
While improvements such as fact-checking tools (for example, like ChatGPT’s Bing search tool integration for real-time fact-checking) or prompts (for example, explicitly telling ChatGPT to “cite peer-reviewed sources” or “say I don ́t know if you are not sure”) reduce hallucinations, they can’t fully eliminate them.
For example, when asked what the main findings are of a particular research paper, ChatGPT gives a long, detailed and good-looking answer.
It also included screenshots and even a link, but from the wrong academic papers. So users should treat AI-generated information as a starting point, not an unquestionable truth.
5. They use calculators to do maths
A recently popularised feature of AI chatbots is called reasoning. Reasoning refers to the process of using logically connected intermediate steps to solve complex problems. This is also known as “chain of thought” reasoning.
Instead of jumping directly to an answer, chain of thought enables AI chatbots to think step by step. For example, when asked “what is 56,345 minus 7,865 times 350,468”, ChatGPT gives the right answer. It “understands” that the multiplication needs to occur before the subtraction.
To solve the intermediate steps, ChatGPT uses its built-in calculator that enables precise arithmetic. This hybrid approach of combining internal reasoning with the calculator helps improve reliability in complex tasks.
- is a Postdoctoral Researcher, Cluster of Excellence “Machine Learning”, University of Tübingen
- This article first appeared in The Conversation
