Hallucination & grounding

At 3 a.m., a customer asks your support bot whether an enterprise refund is available after 30 days. The bot replies in polished prose:

“Yes. Enterprise customers can request a refund within 30 days, and refunds are processed within two hours. There is also a 60-day extended window.”

It sounds like an answer from someone who has read the policy. It contains a true rule, a made-up service level, and a claim the policy explicitly rejects. A busy reader may approve the whole thing in six seconds. The customer may discover the mistake six weeks later, after finance has to unwind it.

This is the practical problem behind hallucination: a large language model (LLM), a system that generates text by predicting likely token sequences, can produce fluent claims without verifying them. Grounding ties an answer’s claims to evidence you trust, such as a policy document, retrieved passage, database result, or tool response.

Grounding makes an answer auditable. It does not make the source correct, retrieval complete, or the model obedient. Those distinctions are where reliable systems begin.

The mental model: fluency is not a truth test

During generation, the model repeatedly chooses the next token given the conversation and text already produced. Its pretraining objective is roughly to produce continuations resembling its training data. This creates a useful writing machine, not an internal librarian checking every sentence against an authoritative record.

If asked, “What is the refund window?” the model may have seen many examples involving refunds, enterprise contracts, 30-day windows, and extended grace periods. If the specific policy is absent, ambiguous, or buried in context, it can assemble a plausible sentence from those patterns. The result may be grammatically perfect and factually empty.

The model is not necessarily deceiving anyone. It is satisfying a prediction problem under uncertainty. “I do not know” is one possible continuation; a specific number common in similar text may be more probable.

Temperature changes sampling variety, not evidence. Higher temperature can make unusual mistakes more likely; lower temperature makes outputs more repeatable. Neither setting checks claims against a source. A deterministic wrong answer is still wrong.

Retrieved text changes the probabilities of possible continuations, but ordinary generation can still blend context with learned patterns. “Use only the supplied sources” is an instruction, not a formal logical constraint.

Grounded, correct, and complete are different properties

Take this policy as the trusted source:

Enterprise customers may request a refund within 30 days of the invoice. Requests after 30 days are prorated according to the remaining term. Approved refunds are normally processed within two business days. There is no extended 60-day window.

Separate these properties:

• Correctness: whether a claim is true in the real world. An outdated policy can support a claim that is no longer correct.
• Grounding: whether the supplied source supports the claim. “Requests after 30 days are prorated” is grounded because the policy says so.
• Completeness: whether the answer included important supported facts. “Refunds are available” may be grounded while omitting the 30-day boundary and proration rule.
• Retrieval quality: whether the system found the right evidence. Generation cannot repair an old or irrelevant document that retrieval supplied.

RAG helps by fetching material at answer time rather than relying only on model memory. But “retrieved” is not “used,” and “used” is not “used correctly.” Grounding is a target for the whole pipeline, not a property guaranteed by adding a vector database.

Faithfulness: inspect the answer one claim at a time

Faithfulness measures how well an answer stays supported by its source. A useful strict version is:

faithfulness = grounded claims / total claims

The denominator contains atomic claims: small statements that can be checked independently. “The refund is available within 30 days and is processed within two hours” contains two claims. Treating it as one hides a partial failure.

Label each claim:

• Grounded: directly supported by the source.
• Inferred: plausible or based on a common assumption, but not supported by the source.
• Contradicted: incompatible with what the source says.

The bot’s quoted answer contains three explicit factual propositions:

1. Enterprise customers can request a refund within 30 days of the invoice.
2. Refunds are processed within two hours.
3. There is a 60-day extended window.

The opening “Yes” is not counted as a separate atomic claim here. By itself, it does not say whether the answer is relying on the 30-day rule, the alleged 60-day window, or some other basis for saying yes. In a production rubric, make that implied answer explicit before scoring it.

Claim one is grounded. Claim two is inferred: the policy says “within two business days,” not two hours. Claim three is contradicted because the policy says there is no extended 60-day window.

The source also says that requests after 30 days are prorated, but the bot did not say that. That omission is a completeness failure, not a faithfulness failure: completeness asks whether important supported facts were included; faithfulness asks whether the claims actually made were supported.

Strict faithfulness is 1 / 3 = 33%. The labels matter more than the percentage: they tell an engineer what failed.

There is no universal faithfulness rubric. Some evaluators allow partial support; others ask whether the whole response is entailed by the context. State your rubric before comparing systems. Otherwise a response that hides four claims in one sentence can appear better than one that exposes them.

An empty answer has zero claims, so the formula is 0 / 0, or N/A. Do not treat that as excellent. Measure abstention correctness and usefulness separately.

# Faithfulness = grounded claims / total claims.
answer_claims = [
    ("Enterprise refunds within 30 days of invoice", "grounded"),
    ("Refunds processed within 2 hours",             "inferred"),
    ("There's a 60-day extended window",             "contradicted"),
]

total = len(answer_claims)
grounded = sum(1 for _, label in answer_claims if label == "grounded")
contradicted = sum(
    1 for _, label in answer_claims if label == "contradicted"
)

faithfulness = grounded / total if total else None

if faithfulness is None:
    print("faithfulness (strict) = N/A (no claims)")
else:
    print(f"faithfulness (strict) = {grounded}/{total} = {faithfulness:.0%}")

print(f"contradictions = {contradicted}")

faithfulness (strict) = 1/3 = 33%
contradictions = 1

A contradiction should usually count as more serious than a missing harmless detail. Keep claim labels and severity rather than only the average.

The evaluator is not an oracle

Frameworks such as RAGAS, TruLens, and DeepEval can automate claim extraction and support judgments, often with another LLM. That helps with regression testing, but moves part of the measurement problem into the judge. It may accept a plausible inference, miss a negation or unit, confuse topical overlap with entailment, or split claims differently from your rubric.

Entailment means the source makes the claim follow, not merely that it mentions related words. “The policy mentions a two-day processing period” does not support “every refund arrives within two hours.” Dates, quantities, negations, and conditions need explicit checks.

Measure answerability alongside faithfulness. A system that always says “I don’t know” has no claims to score, but fails useful, answerable questions. The target is to answer when evidence is sufficient and abstain when it is not.

A production pattern that catches errors

No single prompt is enough. Use separate gates:

1. Define and authorize the source boundary

Identify authoritative documents and store their source ID, effective date, tenant or region, and permissions. Filter by user, tenant, region, and document ACL before retrieval, and repeat authorization checks before tool calls. Never put unauthorized text in context. If authorization is uncertain, deny and log the decision.

Define unanswerable questions too. If the policy says nothing about two-hour processing, the answer should state the gap: “The policy specifies two business days, but does not state an hourly guarantee.”

2. Retrieve evidence with identity intact

Retrieve enough surrounding context to preserve conditions and exceptions, with stable source IDs. Evaluate retrieval separately from generation: was the right document found, did the relevant passage reach context, and did the answer cite it? See RAG evaluations.

3. Require bounded claims and evidence

Ask the model to stay within the source boundary, abstain when support is missing, and provide a source ID or span for each material claim. A useful shape is an answer sentence, supporting source and excerpt, and an uncertainty or missing-evidence note. Structured outputs can reduce formatting errors, but valid JSON can still contain an invented deadline.

4. Check claims after generation

Extract atomic claims and record their text, cited source, entailment status, conflicts, and severity. Use databases or rules for dates, amounts, product IDs, permissions, and statuses where possible. Use an LLM judge for linguistic entailment when necessary, then sample decisions for human review.

Track grounded-claim rate, contradiction rate, citation coverage, answerability, and abstention quality. Citation coverage is the fraction of claims with citations; it is not the fraction whose citations actually support them.

5. Choose an action

Route results instead of always retrying:

• supported and low risk: answer;
• supported but high risk: answer with evidence or require review;
• unsupported but clarifiable: ask a question;
• unsupported or contradicted: abstain and explain the gap;
• conflicting sources: surface the conflict.

A contradiction about a 60-day refund window should block an automatic answer and route the case to finance.

Common failure modes

Empty or irrelevant context, specific answer: the model’s learned prior is filling the evidence gap. Log retrieval results and add an answerability gate that requires abstention or clarification when no relevant passage is found.

Citations that only share the topic: the model attached a source after composing the answer. Validate each claim against its cited span, especially negation, units, dates, and words such as “only,” “always,” and “up to.”

Faithfulness rises while usefulness falls: the system is rewarded for making fewer claims, so it refuses answerable questions or copies passages. Evaluate both answerable and unanswerable questions, rewarding supported answers and correct abstentions separately.

A current-looking citation points to an old rule: semantic retrieval found a real but stale document. Make version and effective date part of retrieval, validation, and incident records. Grounding to yesterday’s truth is still today’s incident.

When grounding is the wrong tool

Grounding suits document-based questions: policies, manuals, contracts, release notes, and procedures. Use a database or deterministic service for balances, inventory, permissions, prices, and transaction status; use a calculator for arithmetic; use a rules engine for high-consequence legal or financial decisions. The LLM can translate a request into a tool call and explain the result, but should not improvise it from prose.

Strict grounding has costs: claim extraction and evidence checks add latency and model calls, citations can make answers awkward, narrow source boundaries can cause refusals, and human review does not scale like token generation. Spend these checks where errors matter, using cheaper deterministic checks when possible.

Faithfulness is not a universal quality score. Creative writing should not cite every invented dragon; a support bot should not invent a refund window.

In one breath

• LLMs predict plausible continuations, not verified facts.
• Grounding means a claim is supported by a chosen source; it does not prove the source is current or true.
• Split answers into atomic claims and label them grounded, inferred, or contradicted. Strict faithfulness is grounded claims divided by total claims; with zero claims, report N/A.
• Track retrieval quality, correctness, completeness, citation coverage, and faithfulness separately.
• Preserve source identity, enforce authorization, validate evidence, detect contradictions, and abstain when support is missing.
• The most dangerous hallucination is often a plausible detail the source never said.

Quick check

Next

Faithfulness is one part of a broader LLM evals program. For retrieval-specific tests, see RAG evaluations. When a single model is too expensive for every check, model routing can reserve stronger evaluators for high-risk answers.