Hallucination & grounding

A hallucination isn’t the model “lying” — it’s the model doing exactly what it was built to do: predict plausible next tokens. When it doesn’t know, it generates something that sounds right, fluently and confidently. That’s why hallucination is so dangerous: the wrong answer looks identical to the right one. The defense is grounding — tying answers to a source — and the skill is measuring whether that grounding actually held.

What “grounded” means

An answer is grounded when its claims are supported by a source you trust (the retrieved context in RAG, a tool result, a document). RAG is grounding by construction — but grounding is a goal, not a guarantee. A model can still ignore the context, over-read it, or blend it with memorized (and wrong) priors. So you have to check.

Measure faithfulness, don’t eyeball it

The reliable method: decompose the answer into atomic claims, then check each one against the source and label it:

• Grounded — directly supported by the source.
• Inferred — plausible, but not actually in the source. The sneaky one.
• Contradicted — conflicts with the source. The dangerous one.

Faithfulness is the fraction of claims that are grounded. Decompose one fluent, confident answer this way and watch how much of it turns out to be air — two claims supported, one quietly invented, one outright wrong:

That decomposition is exactly what faithfulness metrics in RAG eval frameworks (RAGAS, TruLens, DeepEval) automate — usually with an LLM judge that extracts claims and verifies each against the retrieved context.

Reducing hallucination

• Ground it. Retrieval + “answer only from the context; if it’s not there, say you don’t know” is the single biggest lever.
• Ask for citations. Make the model cite which chunk supports each claim — uncited sentences are your hallucination suspects, and it makes QA trivial.
• Constrain when you can. For structured facts, constrained decoding and tool calls beat free-form prose.
• Calibrate uncertainty. Encourage “I don’t know” over a confident guess; measure how often the model abstains when it should.

# Faithfulness = grounded claims / total claims.
answer_claims = [
    ("Enterprise refunds within 30 days of invoice", "grounded"),
    ("Refunds prorated after 30 days",               "grounded"),
    ("Refunds processed within 2 hours",             "inferred"),     # not in source
    ("There's a 60-day extended window",             "contradicted"), # conflicts!
]

total = len(answer_claims)
grounded = sum(1 for _, lab in answer_claims if lab == "grounded")
contradicted = sum(1 for _, lab in answer_claims if lab == "contradicted")

print(f"faithfulness (strict) = {grounded}/{total} = {grounded/total:.0%}")
print(f"contradictions = {contradicted}  <- any contradiction is a red flag")
print("\nThe 'inferred' claim reads fine but isn't supported — exactly where")
print("a fluent hallucination hides. Look at per-claim labels, not just the score.")

faithfulness (strict) = 2/4 = 50%
contradictions = 1  <- any contradiction is a red flag

The 'inferred' claim reads fine but isn't supported — exactly where
a fluent hallucination hides. Look at per-claim labels, not just the score.

The answer reads as four confident sentences, but only two are actually in the source. A holistic “looks good” glance scores it near-perfect; claim decomposition scores it 50% and flags the one contradicted claim as a red flag. The number isn’t the point — the per-claim labels are.

In one breath

• A hallucination isn’t lying — it’s the model predicting plausible tokens when it doesn’t know, so the wrong answer looks exactly like the right one.
• An answer is grounded when its claims trace to a trusted source; RAG grounds by construction but a model can still ignore or over-read the context, so you must check.
• Measure faithfulness by decomposing into atomic claims and labelling each grounded / inferred / contradicted — faithfulness is the grounded fraction.
• Cut hallucination by grounding (“say you don’t know”), demanding citations, constraining structured outputs, and rewarding abstention.
• The 2026 blind spot: scalar scores blur inferred (plausible-but-unsupported) into the number — for high-stakes outputs read the per-claim labels, not the percentage.

Quick check

Next

Grounding metrics are a specialized branch of LLM evals, and they pair directly with RAG evaluations. To cut cost while keeping quality high, see model routing.