Few-shot and chain-of-thought

Two patterns survived successive model upgrades: few-shot prompting (show, don’t tell) and chain-of-thought (think before answering). Both work because they exploit the autoregressive loop — you’re putting helpful tokens into the context that bias the next-token prediction in your favor.

Few-shot: showing examples

Zero-shot means you just describe the task. Few-shot means you also include 2-5 worked examples. The model pattern-matches and continues the pattern.

# Few-shot formatted for the modern messages API.
# Each example is a separate user/assistant turn — the model treats
# the assistant turns as authoritative examples of correct behaviour.

messages = [
    {"role": "system", "content":
        "You convert natural-language times to 24-hour ISO format. "
        "Output only the time string, nothing else."},

    # Example 1
    {"role": "user", "content": "quarter past three in the afternoon"},
    {"role": "assistant", "content": "15:15"},

    # Example 2
    {"role": "user", "content": "ten to nine at night"},
    {"role": "assistant", "content": "20:50"},

    # Example 3
    {"role": "user", "content": "noon"},
    {"role": "assistant", "content": "12:00"},

    # The actual query
    {"role": "user", "content": "half past seven in the morning"},
]

# In production:
#   response = client.messages.create(model="claude-opus-4.7", messages=messages)
# The three demonstrations pin the format, so the model answers in kind.
print("model output:", "07:30")

model output: 07:30

Two things to notice:

1. Each example is a separate turn in the messages array, not concatenated into one big prompt. This is the modern idiom — both the OpenAI Responses API and Anthropic Messages API expect it.
2. The assistant turns show the exact format you want — terse, no prose, no markdown. The model copies that style.

When few-shot beats zero-shot

• Niche output formats the model hasn’t seen often (custom DSLs, internal log formats, weird date conventions).
• Domain-specific labels (“classify as P0/P1/P2 based on our on-call docs”).
• Tone matching — examples of “our brand voice” responses help the model match it.
• Edge case anchoring — include an example of the tricky case so the model knows how to handle it.

When few-shot doesn’t help

• Strong tasks the model already does well (summarization, general Q&A). Examples just eat context tokens.
• When your examples conflict with each other. Three examples with slightly different formats produce slightly different outputs.
• When you have so many examples that you should be fine-tuning instead. Past ~20 examples, fine-tuning is usually cheaper at inference time.

Chain-of-thought: reasoning before answering

Chain-of-thought (CoT) — a prompting technique where you ask the model to reason step-by-step before committing to a final answer — means asking the model to show its work. This works because the model uses those intermediate tokens as scratch space: when it generates the final answer, it can attend to every reasoning step it just emitted, giving it more context to get the answer right.

Without CoT:

Q: A train leaves at 2:15 PM and arrives at 5:03 PM. How long is the trip?
A: 2 hours 45 minutes  ← often wrong on non-round numbers

With CoT:

Q: A train leaves at 2:15 PM and arrives at 5:03 PM. How long is the trip?
A: From 2:15 to 5:15 would be 3 hours, but we stop at 5:03 —
   3 hours minus 12 minutes = 2 hours 48 minutes.

The mechanical reason CoT helps: each token’s prediction can attend to all previous tokens. By emitting the intermediate steps, the model gets to condition its final answer on its own reasoning.

When CoT helps

• Multi-step math — arithmetic with multiple operations.
• Multi-hop reasoning — “the author of book X went to school where…” requires connecting facts.
• Logical deduction — anything that benefits from intermediate conclusions.
• Long-context analysis — extracting from multi-document inputs often benefits from “first identify which docs are relevant”.

When CoT doesn’t help (or hurts)

• Simple lookups. “What’s the capital of France?” doesn’t need reasoning. CoT just adds latency and tokens.
• Classification with structured outputs. If you’re already forcing a schema, asking for reasoning conflicts with terse output.
• When the model would just rationalize a wrong answer. CoT doesn’t make the model correct — it just makes wrong answers more confident-sounding.

Hybrid: reasoning models vs prompting CoT

You have two ways to get reasoning:

1. Reasoning models (o-series, R1-style, Claude with extended thinking) do internal CoT before you see any output. You don’t add “think step by step” — they already are, and manual CoT can even interfere. They cost more and have higher latency, tuned by a thinking budget.
2. Standard models with CoT prompting — cheaper, faster, you control how much reasoning happens.

Rule of thumb: if the task is hard for humans (math, code, multi-step planning), use a reasoning model. For everything else, a standard model with a well-structured prompt is faster and cheaper.

The honest limitations

Few-shot and CoT are real tools, not magic:

• Few-shot prompts add latency and cost — every example is in every request.
• CoT outputs are longer, so they’re slower and more expensive.
• Neither fixes a model that just doesn’t know the answer. They shape the response; they don’t add knowledge.

In one breath

• Few-shot = show 2-5 worked examples as separate user/assistant turns; the model copies the demonstrated format.
• Reach for it on niche formats, domain labels, tone, and edge cases — mine examples from your eval failures, not your typical inputs.
• Chain-of-thought = ask the model to reason before answering; the intermediate tokens become scratch space the final answer can attend to.
• CoT helps on multi-step math, multi-hop reasoning, and deduction; it just costs tokens on simple lookups, and it can make a wrong answer sound confident.
• Both shape the response, neither adds knowledge — and a reasoning model already does CoT internally, so don’t tell it to “think step by step.”

Quick check

Next

The next lesson covers tool calling — giving the model functions it can invoke. That’s the foundation under every agent and the right fix when prompting alone can’t help (math, dates, anything live).