Self-improving agents

Most agents don’t get better with use — same weights, same skills, every run. A self-improving agent does: it closes a loop where it generates attempts, evaluates them, keeps and learns from the good ones, and repeats — bootstrapping its own capability with little or no new human labelling. It’s a powerful idea, and it hinges on one thing: a reliable way to tell good from bad.

The loop, and the linchpin

Every self-improvement method is the same loop with different parts:

The evaluator is the linchpin. With a trustworthy signal — unit tests pass, the math answer is correct, a fitness score — the loop is sound: it only ever keeps verified-good work. Without one, “improvement” is just the model reinforcing its own preferences, which drifts or collapses.

The methods

• STaR (Self-Taught Reasoner). Give the model problems with known answers. It generates chain-of-thought rationales; keep only the rationales that reach the correct answer (the answer is the verifier), fine-tune on them, and repeat. For problems it got wrong, rationalize — show it the answer and ask it to produce a justifying rationale, then keep that too. From a handful of labelled answers, the model bootstraps a large set of good reasoning traces and steadily improves.
• Evolutionary program search (AlphaEvolve). An LLM proposes and mutates programs; an automatic evaluator scores them; the best survive and seed the next generation. Over many rounds it discovers novel, sometimes superhuman algorithms — the evolutionary planning engine pointed at code.
• AI Scientist / automated research. Agents generate hypotheses, run the experiments, analyze results, and write them up — closing the research loop, with experimental results as the verifier.

It bootstraps toward the verifier’s ceiling

Because the loop only retains verified-good work, capability climbs — but only as high as the signal allows. A simple model of the bootstrap makes the shape clear:

acc, ceiling = 0.40, 0.85         # ceiling = the limit the verifier/data can certify
for rnd in range(1, 6):
    acc = acc + 0.5 * (ceiling - acc)   # keep verified-good work, fine-tune, improve a fraction
    print(f"round {rnd}: accuracy {acc:.1%}")

round 1: accuracy 62.5%
round 2: accuracy 73.8%
round 3: accuracy 79.4%
round 4: accuracy 82.2%
round 5: accuracy 83.6%

The agent improves fast at first, then plateaus at the ceiling the verifier can certify. Push past the point where your signal is trustworthy and improvement turns into reward hacking — the model games the metric — or model collapse, where training on its own unfiltered output erodes diversity. Self-improvement is only as good as the verifier behind it.

In one breath

• A self-improving agent closes a loop — generate → evaluate → keep the good → learn → repeat — bootstrapping capability with little new human labelling.
• The verifier is the linchpin: a trustworthy signal (tests pass, correct answer, fitness) is what makes the loop sound; without it, “improvement” drifts.
• STaR keeps only rationales that reach the right answer and fine-tunes on them; AlphaEvolve evolves programs scored by an evaluator; AI-Scientist closes the research loop with experiments as the verifier.
• Capability bootstraps toward the verifier’s ceiling then plateaus (the demo: 40% → ~84% toward an 85% ceiling); push past a trustworthy signal and you get reward hacking or model collapse.
• It comes in weights (STaR), memory (skill libraries), and per-attempt (reflection) forms; recursive self-improvement is the frontier — and a safety concern.

Quick check

Next

Self-improvement appears as memory in skill libraries, as search in HTN & evolutionary planning, and per-attempt in reflection. Bounding an agent that can change itself is the job of agent safety controls.