Are AI hallucinations always bad?
No. A hallucination is the model generating something not grounded in fact, and whether that is bad depends entirely on the use. It is a feature for creative work and dangerous for anything factual, with the worst case being an answer that looks right but is wrong in context.
AI Evaluation & ReliabilityRAG & RetrievalAI Engineering
1. For creative work, hallucination is the feature
When you want a poem, a name, a fresh angle, the model inventing something outside its training is exactly what you came for. Chip Huyen is blunt about it: hallucinations are going to be really great for creative use cases. There, the invention is the goal.
2. For factual work, it is the bane of your existence
The moment the task depends on being correct, the same behavior becomes the risk you spend all your time fighting. A hallucination is bad specifically when it is factually inconsistent with what is supposed to be true. Most enterprise use cases live here.
3. The dangerous one: looks right, isn’t
Vivienne Zhang’s example, which she credits to Jensen Huang at GTC, sticks. Ask an NVIDIA copilot “what is CTL” and it returns a generic, textbook expansion of the acronym. But for a chip designer inside NVIDIA the right answer is the internal one, “Compute Trace Library.” The response is confident, plausible, and wrong in context. For a new employee, that is just as insidious as a completely wrong answer, because it looks right.
Why it matters
“Stop the model from hallucinating” is the wrong goal. The right goal is to know which mode you are in. If you are doing creative work, let it run. If you are doing factual work, your real job is grounding it and catching the answers that look right but are not.
Hear it from the guest
“So I think hallucinations is going to be really, really great things for creative use cases. It's a bad thing for like use cases that depend on factual consistency.”
“The application gives an answer that looks correct, but it's not what the user is looking for, and I will consider that a hallucination.”
Quotes lightly edited to remove filler words.
From the conversation
This explainer is drawn from these episodes — each carries its full transcript.
