Natural language inference

Natural language inference (NLI) — also called textual entailment — asks a sharp question: given a premise and a hypothesis, does the hypothesis follow from the premise? The answer is one of three labels — entailment (it follows), contradiction (it can’t be true), or neutral (could go either way). It looks simple. Yet it is one of the cleanest tests of whether a model understands meaning rather than just matching words.

The three-way judgment

def nli(premise, hypothesis):                  # toy heuristic — real NLI uses a trained model
    p, h = set(premise.lower().split()), set(hypothesis.lower().split())
    if "not" in h and (h - {"not"}) <= p: return "contradiction"
    if h <= p:                            return "entailment"
    return "neutral"

pairs = [
    ("a man is playing guitar", "a man is playing guitar"),       # the hypothesis is stated
    ("a man is playing guitar", "a man is not playing guitar"),   # negated
    ("a man is playing guitar", "a woman is singing"),            # unrelated
]
for prem, hyp in pairs:
    print(f"P={prem!r}  H={hyp!r} -> {nli(prem, hyp)}")

P='a man is playing guitar'  H='a man is playing guitar' -> entailment
P='a man is playing guitar'  H='a man is not playing guitar' -> contradiction
P='a man is playing guitar'  H='a woman is singing' -> neutral

That toy heuristic gets these three right by word overlap and a not check, but it is a fraud. It would call “a person plays an instrument” neutral (no word overlap) when it’s really entailment, and it has no idea that “the man is asleep” contradicts “the man is playing.” Real NLI needs semantics: synonyms, negation scope, quantifiers, and world knowledge. That’s exactly why it became a key benchmark (the datasets SNLI and MNLI) for sentence understanding.

Modeling it — and a cautionary tale

NLI is sentence-pair classification. The strong approach is a cross-encoder: feed both sentences together — [premise] [SEP] [hypothesis] — into a BERT-style model so it can attend across the pair, then classify the three labels. (A bi-encoder, which turns each sentence into its own vector separately, is faster but weaker, since the two sentences never see each other.)

Why it still matters: entailment checks groundedness

NLI’s “does B follow from A?” is a reusable primitive, and it found a second life with LLMs. Faithfulness checking: does the model’s generated answer actually follow from the retrieved source, or did it hallucinate? That’s an entailment question — treat the source as the premise and each generated claim as a hypothesis, and an NLI model flags claims that aren’t entailed. The same primitive powers automated fact-checking (does the evidence entail the claim?) and zero-shot classification (classifying with no labeled training examples, by turning each label into a hypothesis such as “this text is about sports”).

In one breath

• NLI (textual entailment) labels a (premise, hypothesis) pair as entailment, contradiction, or neutral — a clean probe of meaning, not word overlap.
• Surface heuristics fail: real NLI needs synonyms, negation scope, quantifiers, and world knowledge — which is why SNLI/MNLI became key understanding benchmarks.
• It’s sentence-pair classification, best done with a cross-encoder (premise [SEP] hypothesis into BERT) that attends across the pair.
• It’s the textbook dataset-artifact cautionary tale — models scored well by reading only the hypothesis (negation tells leaked the label), learning the shortcut, not entailment.
• The entailment primitive now checks LLM faithfulness/groundedness (does the answer follow from the source?), powers fact-checking, and enables zero-shot classification.

Quick check

Next

NLI’s entailment check underpins faithfulness in hallucination & grounding and rigorous agent benchmarks. As sentence-pair classification it builds on text classification with a BERT cross-encoder.