Structured outputs with JSON Schema

You ask the model “extract the user’s name, age, and city from this text” and get back:

Sure! Here's what I found:

Name: Sara Kim
Age: 29
Located in San Francisco (sometimes works remotely from Tokyo)

Now write the regex. Then handle “twenty-nine” instead of 29. Then handle the cases where the model adds disclaimers, refuses, or uses a different key. This is the problem structured outputs solve.

The modern shape

Both major APIs now support constrained decoding: you give them a JSON Schema, and the model is forced (at the token level) to emit only tokens that keep the output valid. Not “asked nicely” — actually constrained. (Mechanically: during generation, any token that would make the partial output invalid according to the schema gets its probability set to zero before sampling.) The mechanics — token masking, grammar compilation, and the near-zero-overhead engines (XGrammar, llguidance) behind it — are covered in depth in constrained decoding.

• OpenAI: a JSON Schema in the request (response_format on Chat Completions, text.format on the Responses API), or the client.responses.parse(...) helper with a Pydantic model.
• Anthropic: a JSON Schema in output_config.format, or the client.messages.parse(...) helper with a Pydantic model.
• Gemini: response_mime_type="application/json" plus response_schema.

Pydantic as the source of truth

In Python, you write the schema once as a Pydantic model and the SDKs convert it. Your schema, your types, and your validation are the same object.

from pydantic import BaseModel, Field
from typing import Literal

class Person(BaseModel):
    name: str = Field(description="Full name of the person")
    age: int = Field(ge=0, le=150)
    city: str
    employment_status: Literal["employed", "unemployed", "student", "retired"]

schema = Person.model_json_schema()           # Pydantic emits a JSON Schema
print("fields:", list(schema["properties"].keys()))
print("age:", schema["properties"]["age"]["minimum"], "to", schema["properties"]["age"]["maximum"])
print("status options:", schema["properties"]["employment_status"]["enum"])

fields: ['name', 'age', 'city', 'employment_status']
age: 0 to 150
status options: ['employed', 'unemployed', 'student', 'retired']

That schema — field names, the integer range on age, the four allowed employment_status values — is exactly what gets sent to the API. The model literally cannot emit {"age": "twenty-nine"}, because at the moment it is filling age the constrained decoder masks every token that is not a digit:

The shape of the API call

from pydantic import BaseModel, Field
from typing import Literal
from openai import OpenAI

class Person(BaseModel):
    name: str
    age: int = Field(ge=0, le=150)
    city: str
    employment_status: Literal["employed", "unemployed", "student", "retired"]

client = OpenAI()

# The SDK sends Person's JSON Schema to the model, then parses the reply back
# into a validated Person — malformed JSON or an out-of-range age is rejected for you.
response = client.responses.parse(
    model="gpt-4.1",
    input="Extract the person's details from this text: "
          "'Sara Kim is a 29-year-old software engineer living in San Francisco.'",
    text_format=Person,
)

person = response.output_parsed          # a validated Person instance
print(person.name, person.age, person.city, person.employment_status)
# -> Sara Kim 29 San Francisco employed

The flow is: Pydantic model → JSON Schema → API → JSON response → back into Pydantic. One schema, end to end.

What can go wrong

Constrained decoding handles the format. It does not handle:

• Wrong content. The model can still hallucinate the wrong name or the wrong age. Constrained decoding enforces the JSON structure (types, required keys, enum values) — it does not enforce Pydantic validators like ge=0, le=150. Those run on your side after you call model_validate_json.
• Missing nuance. If the text says “around 30”, the schema forces an int. The model has to pick.
• Refusals. The model might emit {"name": "[REFUSED]", ...} if it can’t find an answer. Always handle this case.

Structured outputs with Claude

Claude’s Messages API has native structured outputs. You hand it a JSON Schema in output_config.format, and the reply is guaranteed to be JSON that matches it. The Python SDK’s messages.parse helper takes a Pydantic model directly and gives you back a validated instance, mirroring the OpenAI call above:

# Claude native structured outputs (read-only here; needs ANTHROPIC_API_KEY)
# response = client.messages.parse(
#     model="claude-opus-5-5",
#     max_tokens=1024,
#     messages=[{"role": "user", "content": prompt}],
#     output_format=Person,             # the same Pydantic model as above
# )
# person = response.parsed_output       # a validated Person instance

Without the helper, pass the schema yourself as output_config={"format": {"type": "json_schema", "schema": ...}} and read the JSON from the text block of the reply.

The older idiom: forcing a tool call

Before native structured outputs existed, the standard trick was to declare a single tool whose input_schema is your output schema and force the model to call it with tool_choice. You will still see it in older code and in libraries such as instructor and LangChain’s with_structured_output:

# The tool-forcing idiom (works on older Claude models)
tools = [{
    "name": "extract_person",
    "description": "Extract the person's details from the input text.",
    "input_schema": Person.model_json_schema(),
}]

# response = client.messages.create(
#     model="claude-opus-4-7",
#     max_tokens=1024,
#     tools=tools,
#     tool_choice={"type": "tool", "name": "extract_person"},  # force this tool
#     messages=[{"role": "user", "content": prompt}],
# )
# person = Person.model_validate(response.content[0].input)

When NOT to use structured outputs

• Long-form generation (essays, summaries, code). Forcing a schema on prose makes the prose worse.
• When the model genuinely needs to refuse or escalate. A schema that requires every field be filled removes the model’s option to say “I don’t know”.
• For one-off prototypes. Setting up the schema can be more work than parsing the output for a 5-line script.

For everything else — extraction, classification, routing, function arguments — structured outputs should be your default.

In one breath

• Parsing free-text LLM output is a losing battle — constrain the format instead.
• One Pydantic model becomes the JSON Schema you send and the validator you parse back into — no drift.
• Constrained decoding masks any token that would break the schema, so the JSON is valid by construction.
• OpenAI uses responses.parse (or a schema in the request); Claude uses output_config.format or messages.parse. Forcing a single tool is the older idiom, and the newest Claude models reject it. Always set strict: true.
• It enforces structure, not truth or Pydantic validators (ge/le) — those run on your side. Don’t use it for prose.

Quick check

Next

The next lesson covers prompt patterns — the techniques that actually move the needle on quality, and the cargo-cult ones that don’t.