Evaluator-optimizer (generate, critique, revise)

The evaluator-optimizer pattern — older lessons and a lot of blog posts still call it “reflection” — is Anthropic’s name for the closest thing in agent design to “ask the model to proofread itself.” You generate a draft, ask an LLM to evaluate it, then ask the model to revise based on that evaluation. The whole thing fits in three calls.

It works because LLMs are better at recognizing errors than avoiding them. The first pass — the generator — is the model trying to be right. The evaluator pass is the model wearing a different hat, reading the draft as input rather than producing it — and that second look often catches mistakes the draft missed.

The shape of the loop

1. Generator — "Answer this question: ..."
2. Evaluator — "Here's the answer above. What's wrong with it?"
3. Generator — "Here's the original answer and the critique. Produce a better version."

That’s the basic recipe. You can run it once, or loop until the evaluator says “looks good” (with a hard cap so you don’t burn money). Most production uses do exactly one revision pass — diminishing returns kick in fast.

Same model or two different models?

The generator and evaluator can be the same model with different prompts, or two different models entirely. The trade-offs:

• Same model, different prompts — the default. Cheapest, simplest, one provider. Works because the critique prompt frames the model differently (it reads the draft as input, not output). Good for writing polish, code style, and most production cases.
• Different models (smaller evaluator) — use a fast cheap model as the evaluator. Saves money if you iterate many times. Risk: a weaker evaluator may miss the same flaws the generator missed.
• Different models (stronger evaluator) — use a more capable model as the evaluator. Best for domains where the evaluation criteria are genuinely harder than the generation (formal proofs, security reviews, regulatory compliance).

Start with same-model. Move to a different evaluator only when your evals show the bottleneck is critique quality, not draft quality.

Why a “second look” works

LLMs predict one token at a time. Once the draft starts going somewhere weak, the model commits and keeps generating consistent-but-wrong text. A critique prompt gives the model a fresh start: it reads the draft as input rather than as something it’s producing, and can spot the flaws without being trapped by what it already wrote.

The critic is also usually given different framing — “be picky”, “find errors”, “list at least three problems”. That framing alone shifts the response distribution toward criticism instead of confirmation.

The loop in code

# A reflection loop with a mocked LLM.
# In production, swap mock_llm for client.messages.create(...).

def mock_llm(prompt):
    # Deterministic mock — returns a different response per role. Note the
    # revise check comes FIRST: the revise prompt mentions "critique" too,
    # so checking critique first would wrongly return the critique.
    p = prompt.lower()
    if "revise" in p or "improved version" in p:
        return (
            "Revised proof:\n"
            "Base case: P(1) holds since 1 = 1*(1+1)/2.\n"
            "Inductive step: assume P(k); then 1+2+...+(k+1) = k(k+1)/2 + (k+1)\n"
            "                            = (k+1)(k+2)/2 = P(k+1). QED."
        )
    if "critique" in p or "what's wrong" in p:
        return (
            "1. The proof skips a step at line 3.\n"
            "2. The variable 'n' is used before being defined.\n"
            "3. No base case is shown."
        )
    if "draft" in p or "prove" in p:
        return "Sum of 1..n is n(n+1)/2 because pairs add to n+1."
    return "I don't know."

def reflection_agent(task, max_revisions=1):
    draft = mock_llm(f"Draft an answer. Task: {task}")        # 1. draft
    print(f"DRAFT:\n{draft}\n")

    current = draft
    for i in range(max_revisions):
        critique = mock_llm(                                  # 2. critique
            f"Critique this answer. Be picky and list issues.\n\nAnswer:\n{current}")
        print(f"CRITIQUE {i+1}:\n{critique}\n")
        if "no issues" in critique.lower():
            break
        current = mock_llm(                                   # 3. revise
            f"Here is the original answer and a critique. Produce an improved "
            f"version.\n\nOriginal:\n{current}\n\nCritique:\n{critique}\n\nRevised:")
        print(f"REVISION {i+1}:\n{current}\n")

    return current

final = reflection_agent("Prove that 1+2+...+n = n(n+1)/2 by induction.")
print(f"FINAL ANSWER:\n{final}")

DRAFT:
Sum of 1..n is n(n+1)/2 because pairs add to n+1.

CRITIQUE 1:
1. The proof skips a step at line 3.
2. The variable 'n' is used before being defined.
3. No base case is shown.

REVISION 1:
Revised proof:
Base case: P(1) holds since 1 = 1*(1+1)/2.
Inductive step: assume P(k); then 1+2+...+(k+1) = k(k+1)/2 + (k+1)
                            = (k+1)(k+2)/2 = P(k+1). QED.

FINAL ANSWER:
Revised proof:
Base case: P(1) holds since 1 = 1*(1+1)/2.
Inductive step: assume P(k); then 1+2+...+(k+1) = k(k+1)/2 + (k+1)
                            = (k+1)(k+2)/2 = P(k+1). QED.

The first draft is a one-line hand-wave; the critique (the model reading its own draft as input) names three concrete gaps; the revision fixes them into a real inductive proof. The three calls are all the same model — the difference is just the prompt framing. Critique and revise are separate calls here; you can also collapse them, which is the next variant.

Two ways to wire it

Two-call pipeline (above). Draft is one call, critique-then-revise is one or two more. Pros: each step has its own prompt you can tune. Cons: 3× the latency.

Single-shot draft + critique + revision. Ask the model in one prompt to produce all three sections, with clear delimiters. Output looks like:

<draft>
First pass at the answer.
</draft>
<critique>
Three things wrong with the draft above.
</critique>
<final>
Revised version that fixes the critique.
</final>

Pros: one API call, much cheaper. Cons: the model sometimes phones it in — generates a weak critique because it’s about to revise anyway. You’ll see worse quality than the explicit two-call version on hard tasks.

Use single-shot when latency matters and the task is moderate. Use the multi-call pipeline when quality is the bottleneck and you can afford the tokens.

Self-consistency: a cheaper, often-better cousin

For tasks with a single right answer (math, multiple choice, structured extraction), self-consistency beats reflection on cost and accuracy. The recipe:

1. Sample the same prompt N times at temperature > 0.
2. Extract the answer from each sample.
3. Return the majority vote.

You’re not asking the model to critique itself — you’re asking it the same question many times and trusting the wisdom of its own crowd. On math benchmarks, N=5 with majority voting often outperforms reflection-style revision at lower total cost.

# Self-consistency: ask the same question N times at temperature > 0,
# then majority-vote the final answer.
from collections import Counter

# The 5 samples the model returned for "What is 6 * 7?" — mostly right,
# occasionally wrong (rigged here as a fixed list for a stable demo).
samples = ["42", "41", "42", "42", "43"]

print("samples:", samples)
winner, count = Counter(samples).most_common(1)[0]
print(f"final: {winner} (won {count}/{len(samples)} votes)")

samples: ['42', '41', '42', '42', '43']
final: 42 (won 3/5 votes)

No critique, no revision — just the same question asked five times. Three samples land on 42, one slips to 41, one to 43; the majority vote recovers the right answer the model usually gets. That’s why, for tasks with a single discrete answer, self-consistency often beats reflection on both cost and accuracy. It only works when there’s a discrete answer to vote on. For open-ended writing, reflection is the right tool.

When reflection helps — and when it hurts

It helps when:

• The task is hard enough that the model sometimes gets it wrong on the first try. Complex reasoning, multi-step math, long-form writing, code with subtle bugs.
• You can write a specific critic prompt. “Find logical errors”, “check the JSON schema matches”, “verify all citations exist”.
• The cost of a wrong answer is higher than the cost of 2-3 extra model calls.

It hurts when:

• The task is easy — a one-line lookup or a yes/no classification. Reflection just adds latency and occasionally introduces new errors by over-thinking.
• The critic prompt is generic — “is this good?” rarely surfaces real issues; the model says “looks fine” and you’ve wasted three calls.
• The task needs fresh data. Reflection won’t fix a hallucinated fact. You need tool use (search, database) for that. Don’t confuse the two.

In one breath

• Evaluator-optimizer (a.k.a. reflection): generate a draft, evaluate it, revise — it works because LLMs recognize errors better than they avoid them.
• It’s usually the same model under three prompts; the critique hat reads the draft as input and is framed to find flaws. Move to a separate evaluator only when evals say critique is the bottleneck.
• Wire it as a two-call pipeline (tunable, slower) or single-shot draft+critique+final (cheap, but the critique can be lazy).
• For tasks with a discrete answer, self-consistency (sample N, majority-vote) often beats reflection on cost and accuracy.
• It helps on hard reasoning/writing with a specific critic prompt; it hurts on easy tasks, with generic critics, or for missing facts (use tools) — always A/B against a no-reflection baseline.

Quick check

Next

The next lesson is tools — the schema contract, error handling, and the production patterns that make tool-using agents actually reliable.