An ungrounded answer comes from whatever the model absorbed in training, with no way to see where a claim came from. A grounded answer is built from material put in front of the model at the time of the question, usually by retrieval, and ideally cites which source supports which claim. That gives a reader something to check and gives the model less reason to invent.
Grounding can reduce hallucinations but doesn’t guarantee accuracy. The sources can be wrong or stale, the retrieval can miss the right document, and the model can still go beyond what the sources say. Teams measure the last of these with faithfulness checks, and many enterprise hallucination problems turn out to be data problems.
A faithful answer from the wrong version
An invented case: a company’s knowledge base holds two versions of its travel policy. The 2025 page caps meals at $50 a day; the 2026 page raises the cap to $75. Retrieval returns the 2025 page, and the assistant answers “$50 a day” and cites it.
Every claim in that answer is supported by its source, so a faithfulness check passes. The answer is still wrong for an employee travelling this year. Grounding made the error easy to trace: the citation shows which page was used, and the date on that page shows why it no longer applies. It didn’t prevent the error. Check that the supplied evidence is current and applies to the question, as well as whether the answer stays within it.
Sources
- Lewis et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Names providing provenance for a model's outputs as an open problem for models that rely only on parametric knowledge, and pairs a generator with retrieved documents to address it; checked October 25, 2026.
Go deeper
- What is RAG, and why do AI systems use it? AI, decoded · What Is RAG (Retrieval-Augmented Generation)
- Why do LLMs hallucinate? AI, decoded · Why LLMs Hallucinate
- Are AI hallucinations a data problem or a model problem? AI, decoded · Are AI Hallucinations a Data Problem?
- Are AI hallucinations always bad? AI, decoded · When AI Hallucinations Are Good (and When They're Dangerous)