Guardrails & output validation

An LLM is a probabilistic text generator, but you usually wire it into deterministic systems — a function that expects valid JSON, a UI that must not display a credit-card number, a support bot that should answer about your product and nothing else. Most of the time the model behaves. Occasionally it returns malformed JSON, leaks PII it saw in context, drifts off-topic, or confidently makes something up. A single bad response can crash a parser or violate a policy.

A guardrail is the answer: a validation layer that sits between the model and the rest of your stack, checks every input and output against rules you define, and refuses to pass bad output through. It is the deterministic seatbelt around a probabilistic engine.

Two places to guard

Guardrails sit on both sides of the model:

• Input guardrails run before the call: reject or sanitise requests — strip or flag PII, block disallowed topics, catch jailbreak patterns, enforce length and rate limits. Cheaper to stop a bad request than to clean up a bad answer.
• Output guardrails run after the call, on the response or the tool call it wants to make: validate the format, redact leaked PII, block unsafe content, and check that the answer is actually grounded in the provided context. This is the last line before the output reaches a user or executes an action.

The core mechanic: validate, then react

What makes a guardrail more than a filter is what it does on failure. The loop:

On a failed check a guardrail can fix the output programmatically (e.g. coerce a type), reask the model with the validation error appended (the most common move for format/grounding failures), filter the bad part out (redact PII), or raise and block the response entirely. Here is the reask loop on a structured output that must be valid JSON with an allowed sentiment:

import json
ALLOWED = {"positive", "negative", "neutral"}

def validate(output):                          # an output guardrail
    try:
        obj = json.loads(output)
    except json.JSONDecodeError:
        return False, "not valid JSON"
    if obj.get("sentiment") not in ALLOWED:
        return False, "sentiment not in allowed set"
    return True, "ok"

# The model's two attempts: first malformed, then fixed after the reask
attempts = ['Sure! the sentiment is positive.', '{"sentiment": "positive"}']
for i, out in enumerate(attempts, 1):
    ok, msg = validate(out)
    print(f"attempt {i}: valid={ok}  ({msg})")
    print("  -> accepted" if ok else "  -> reask the model")
    if ok:
        break

attempt 1: valid=False  (not valid JSON)
  -> reask the model
attempt 2: valid=True  (ok)
  -> accepted

The first attempt is chatty prose that breaks the parser; the guardrail catches it, sends the error back, and the model returns clean JSON on the retry. The caller only ever sees validated output — never the malformed first try.

Three tiers of validator

Guardrails differ enormously in cost. A good stack runs the cheap, certain checks first and only spends a model call when it must:

Order them cheapest-first: a malformed JSON should fail at tier 1 for free, never reaching the LLM-judge.

Fail-open or fail-closed?

When a guardrail itself errors or times out, you must choose its default:

• Fail-closed — block the response if the guard cannot confirm it is safe. Right for anything irreversible or high-stakes (financial actions, medical content, posting externally). Safety beats availability.
• Fail-open — let the response through if the guard is unavailable. Acceptable only for low-stakes, non-safety checks where blocking would hurt UX more than a rare miss. Choose this deliberately, never by accident.

In one breath

• An LLM is probabilistic but plugs into deterministic systems; a guardrail is the validation layer that checks every input/output and refuses to pass bad output through.
• Input rails sanitise the request; output rails validate the response or tool call — format, PII, safety, and grounding — as the last line before a user or action.
• The core mechanic is validate → react: on failure, fix, reask (feed the error back — the common case), filter (redact), or raise (block).
• Use three tiers cheapest-first: deterministic (schema/regex, ~free) → ML classifier (PII/toxicity/Llama Guard) → LLM-as-judge (grounding/policy, a full call).
• Decide fail-closed (block when unsure — high stakes) vs fail-open (low-stakes) on purpose; mind the latency cost and reach for Guardrails AI / NeMo Guardrails / Pydantic-Instructor / Llama Guard rather than hand-rolling.

Quick check

Next

Guardrails enforce; LLM evals measure whether they (and the model) actually work. For the adversarial-input subset, see prompt injection; for the grounding rail specifically, see hallucination & grounding.