Hallucinations
Why can AI be confident and wrong?
The model generates statistically likely sequences, rather than querying a database of guaranteed facts.
The model can produce a confident-sounding answer even when no correct answer exists — because of limits in training data, ambiguous prompts, statistical generation, and the lack of guaranteed factual verification. This isn't simply "the AI doesn't know."
In more detail
A hallucination is fluent output that is false or unsupported by facts. The model was trained to produce plausible continuations, and nothing in the basic process checks claims against reality. When information is rare, missing or ambiguous, the model may still produce a confident-sounding guess. This is not lying: the model has no intention to deceive.
Later training stages can reduce hallucinations, but they do not remove them. A confident tone is not evidence that something is correct. Invented references and quotations are a well-known example.
What helps:
- Ask for sources and check them, since a model can invent sources.
- Put the relevant text in the prompt and ask the model to answer from it.
- Use systems that retrieve information from documents (retrieval-augmented generation, RAG) or use tools such as search and calculators.
- Verify important facts, especially in medicine, law, finance and safety.