What is an AI hallucination?

In artificial intelligence, "hallucinating" has nothing psychiatric about it: it is the name that stuck for a false answer delivered with total confidence. An assistant that swears your kettle shuts off by itself when it runs out of water, when it does not, is not lying or failing on purpose; it is completing a sentence with whatever seems most probable. For a store the problem is twofold. The shopper gets a wrong fact about your product without ever setting foot on your site, and you do not see the answer: you find out, if you find out, from a return or a question in the support chat.

How it works

Google explains it in its guide to AI-generated content: generative models do not retrieve data, they predict the most likely sequence of words based on what they learned in training, so what they produce can contain errors, known as hallucinations.⁠^{[1]Google Search Central. Google Search's guidance on using generative AI content on your website} An LLM (large language model) has no box that says "I don't know this fact"; it has more and less probable words.

A paper by Adam Tauman Kalai, Ofir Nachum, Santosh Vempala and Edwin Zhang, published on arXiv on September 4, 2025, gives the most cited explanation of why they do not go away. Models behave like a student facing a hard exam question: when they are not sure, they sometimes guess instead of admitting it. According to the authors, training and evaluations reward guessing over admitting doubt, because most benchmarks score that way, and a model optimized to pass exams learns that answering something pays better than not answering.⁠^{[2]arXiv (Kalai, Nachum, Vempala and Zhang; September 4, 2025). Why Language Models Hallucinate}

The research distinguishes two kinds, and the distinction is useful for a store. A reference survey of the problem calls a hallucination intrinsic when it contradicts the source the model had in front of it, and extrinsic when it adds something that cannot be verified against that source: it neither confirms nor refutes it. The authors warn that the extrinsic kind is not always an error, because it can come from outside information that happens to be correct.⁠^{[3]arXiv (Ji et al.; ACM Computing Surveys). Survey of Hallucination in Natural Language Generation} In plain terms: if the assistant read your product page and says something else, it is intrinsic; if it says something your page does not cover, it is extrinsic, and it may be right or wrong.

AI search, which consults the web before answering, helps, but does not fully fix it. Google says so bluntly about its own summaries: AI Overviews can make mistakes, and it does, and it recommends checking important information in more than one place and opening the supporting links.⁠^{[4]Google Search Help. Find information in faster & easier ways with AI Overviews in Google Search} Anthropic, which makes Claude, lists techniques to reduce them, such as letting the model say "I don't know," backing the answer with direct quotes from the documents, or making each claim carry its citation, and adds that they reduce hallucinations but do not eliminate them entirely.⁠^{[5]Anthropic, Claude documentation. Reduce hallucinations}

In a store

A fictional example: a store sells an electric kettle. A shopper asks an assistant with search and gets a recommendation. Here is how each kind of hallucination looks from the store's side:

The last two rows have something in common: the fact the shopper needed was not written on the page in a way a crawler or an assistant could read. A fact in plain text does not guarantee that no assistant will get it wrong, but it does leave a source to check the answer against, and an extrinsic hallucination is, by definition, one that adds what the source does not say.⁠^{[3]arXiv (Ji et al.; ACM Computing Surveys). Survey of Hallucination in Natural Language Generation}

How to check

1. Pick the five features you get asked about most and write down, for each one, what your product page says. That is the reference you will check against.
2. Ask the assistants about your product, by name, one feature at a time, and note the answer with its date. Repeat the same question in new chats: Anthropic suggests comparing several answers to the same question, because inconsistencies between them can give a hallucination away.⁠^{[5]Anthropic, Claude documentation. Reduce hallucinations}
3. Open the sources each answer cites and look for the fact in them, as Google recommends for AI Overviews.⁠^{[4]Google Search Help. Find information in faster & easier ways with AI Overviews in Google Search} If the source is your page and the fact does not match, it is intrinsic; if the fact is not there, it is extrinsic.
4. Fix your own side first. If the fact is not on your product page, or is only in an image or a PDF, write it as text. If the wrong answer shows up on Google, send your feedback with the thumbs-down that every AI Overview carries.⁠^{[4]Google Search Help. Find information in faster & easier ways with AI Overviews in Google Search}
5. Repeat the check regularly. Clione, for example, asks ChatGPT and Gemini what shoppers ask and links each answer, which is where you read what they say about your products.

Related concept: grounding

Grounding is anchoring a model's answer in specific documents, for example the ones it retrieves from an index with RAG (retrieval-augmented generation), instead of letting it answer only with what it learned. The difference from a hallucination is that of a remedy and a symptom: the hallucination is the failure; grounding is one of the ways to reduce it. Anthropic uses it this way: asking the model to extract direct quotes before answering anchors the answer in the real text and reduces hallucinations, without eliminating them.⁠^{[5]Anthropic, Claude documentation. Reduce hallucinations} An assistant with search does grounding and can still hallucinate, as Google's warning about its own summaries shows.⁠^{[4]Google Search Help. Find information in faster & easier ways with AI Overviews in Google Search}

To keep going, the guide on what an AI crawler sees on your store shows how much of your product page each agent gets to read, and the AI search reading path lays out the rest of the terms.