Structured outputs with JSON Schema

You ask the model “extract the user’s name, age, and city from this text” and get back:

Sure! Here's what I found:

Name: Sara Kim
Age: 29
Located in San Francisco (sometimes works remotely from Tokyo)

Now write the regex. Then handle “twenty-nine” instead of 29. Then handle the cases where the model adds disclaimers, refuses, or uses a different key. This is the problem structured outputs solve.

The modern shape

Both major APIs now support constrained decoding: you give them a JSON Schema, and the model is forced (at the token level) to emit only tokens that keep the output valid. Not “asked nicely” — actually constrained. (Mechanically: during generation, any token that would make the partial output invalid according to the schema gets its probability set to zero before sampling.) The mechanics — token masking, grammar compilation, and the near-zero-overhead engines (XGrammar, llguidance) behind it — are covered in depth in constrained decoding.

• OpenAI: response_format={"type": "json_schema", "json_schema": {...}} or use the helper client.responses.parse(...) with a Pydantic model.
• Anthropic: define a tool with the schema as its input, set tool_choice={"type": "tool", "name": "..."} to force its use.
• Gemini: response_mime_type="application/json" plus response_schema.

Pydantic as the source of truth

In Python, you write the schema once as a Pydantic model and the SDKs convert it. Your schema, your types, and your validation are the same object.

from pydantic import BaseModel, Field
from typing import Literal

class Person(BaseModel):
    name: str = Field(description="Full name of the person")
    age: int = Field(ge=0, le=150)
    city: str
    employment_status: Literal["employed", "unemployed", "student", "retired"]

schema = Person.model_json_schema()           # Pydantic emits a JSON Schema
print("fields:", list(schema["properties"].keys()))
print("age:", schema["properties"]["age"]["minimum"], "to", schema["properties"]["age"]["maximum"])
print("status options:", schema["properties"]["employment_status"]["enum"])

fields: ['name', 'age', 'city', 'employment_status']
age: 0 to 150
status options: ['employed', 'unemployed', 'student', 'retired']

That schema — field names, the integer range on age, the four allowed employment_status values — is exactly what gets sent to the API. The model literally cannot emit {"age": "twenty-nine"}, because at the moment it is filling age the constrained decoder masks every token that is not a digit:

The shape of the API call

from pydantic import BaseModel, Field
from typing import Literal
from openai import OpenAI

class Person(BaseModel):
    name: str
    age: int = Field(ge=0, le=150)
    city: str
    employment_status: Literal["employed", "unemployed", "student", "retired"]

client = OpenAI()

# The SDK sends Person's JSON Schema to the model, then parses the reply back
# into a validated Person — malformed JSON or an out-of-range age is rejected for you.
response = client.responses.parse(
    model="gpt-4.1",
    input="Extract the person's details from this text: "
          "'Sara Kim is a 29-year-old software engineer living in San Francisco.'",
    text_format=Person,
)

person = response.output_parsed          # a validated Person instance
print(person.name, person.age, person.city, person.employment_status)
# -> Sara Kim 29 San Francisco employed

The flow is: Pydantic model → JSON Schema → API → JSON response → back into Pydantic. One schema, end to end.

What can go wrong

Constrained decoding handles the format. It does not handle:

• Wrong content. The model can still hallucinate the wrong name or the wrong age. Constrained decoding enforces the JSON structure (types, required keys, enum values) — it does not enforce Pydantic validators like ge=0, le=150. Those run on your side after you call model_validate_json.
• Missing nuance. If the text says “around 30”, the schema forces an int. The model has to pick.
• Refusals. The model might emit {"name": "[REFUSED]", ...} if it can’t find an answer. Always handle this case.

The Anthropic tool-use trick — the canonical pattern

Anthropic’s Messages API has no separate response_format parameter (unlike OpenAI). Instead, the canonical idiom for structured outputs against Claude is: declare a single tool whose input_schema is your output schema, then force the model to call that one tool with tool_choice:

# The standard structured-outputs idiom for Claude
tools = [{
    "name": "extract_person",
    "description": "Extract the person's details from the input text.",
    "input_schema": Person.model_json_schema(),
}]

# response = client.messages.create(
#     model="claude-opus-4.7",
#     tools=tools,
#     tool_choice={"type": "tool", "name": "extract_person"},  # force this tool
#     messages=[{"role": "user", "content": prompt}],
# )
# # The model cannot reply with prose — it must produce a valid call to
# # extract_person with arguments that match Person's schema.
# person = Person.model_validate(response.content[0].input)

It feels weird the first time — you’re using the tool system as a schema enforcer — but it’s the pattern Anthropic recommends in their docs and it’s the basis of how SDK helpers (instructor, LangChain’s with_structured_output, etc.) achieve typed outputs against Claude under the hood. Some recent Claude models also expose native structured output modes, but the tool-use pattern remains the most portable.

When NOT to use structured outputs

• Long-form generation (essays, summaries, code). Forcing a schema on prose makes the prose worse.
• When the model genuinely needs to refuse or escalate. A schema that requires every field be filled removes the model’s option to say “I don’t know”.
• For one-off prototypes. Setting up the schema can be more work than parsing the output for a 5-line script.

For everything else — extraction, classification, routing, function arguments — structured outputs should be your default.

In one breath

• Parsing free-text LLM output is a losing battle — constrain the format instead.
• One Pydantic model becomes the JSON Schema you send and the validator you parse back into — no drift.
• Constrained decoding masks any token that would break the schema, so the JSON is valid by construction.
• OpenAI uses response_format/responses.parse; Anthropic forces a single tool via tool_choice. Always set strict: true.
• It enforces structure, not truth or Pydantic validators (ge/le) — those run on your side. Don’t use it for prose.

Quick check

Next

The next lesson covers prompt patterns — the techniques that actually move the needle on quality, and the cargo-cult ones that don’t.