Multi-agent debate & voting

A single agent reasons once and commits. If its chain of thought takes a wrong turn, nothing catches it — the same model that made the error is the only one checking it. A simple, powerful fix is to use several agents and combine them. There are two ways to do that, and the difference matters: they can vote (answer independently, then aggregate) or they can debate (see each other’s answers and argue toward a better one).

Two ways to combine agents

• Aggregate (vote). Run several agents independently on the same question, then combine — majority vote for a label, or an aggregator agent for free-form text. The agents never interact. (Sampling one model many times and majority-voting is the familiar special case, self-consistency; here we generalize it across agents.)
• Debate. Agents see each other’s answers and respond — critique, defend, revise — over a few rounds, then converge or are picked by a judge. The interaction is the point.

Why voting helps at all

The reason a vote beats a single agent is the wisdom of crowds: if each agent is better than a coin flip and their errors are independent, the majority is right more often than any individual. The probability is exact — count the ways a majority can be correct:

from math import comb

def majority_correct(n, p):       # n (odd) agents, each correct with prob p, majority vote
    need = n // 2 + 1             # how many must be right for the majority to be right
    return sum(comb(n, k) * p**k * (1 - p)**(n - k) for k in range(need, n + 1))

for n in [1, 3, 5, 7, 9, 15]:
    print(f"{n:>2} agents (each 60% correct): majority correct = {majority_correct(n, 0.6):.1%}")

 1 agents (each 60% correct): majority correct = 60.0%
 3 agents (each 60% correct): majority correct = 64.8%
 5 agents (each 60% correct): majority correct = 68.3%
 7 agents (each 60% correct): majority correct = 71.0%
 9 agents (each 60% correct): majority correct = 73.3%
15 agents (each 60% correct): majority correct = 78.7%

Five mediocre 60%-agents vote their way to 68%; fifteen reach 79%. The catch is the word independent. If every agent is the same model with the same prompt, they tend to make the same mistakes — their errors are correlated, and the vote barely moves. That correlated-error problem is exactly what debate is built to break.

Debate: agents argue toward a better answer

In multi-agent debate, each agent answers, then reads the others’ answers and revises its own, for a few rounds. A confident, well-argued correct answer tends to pull the others toward it; a lone error gets challenged instead of rubber-stamped. After the rounds, a vote or a judge picks the final answer.

This is the idea behind Du et al.’s multiagent debate (several model instances debate to improve factuality and reasoning) and a modern echo of Minsky’s Society of Mind — intelligence emerging from many simple agents interacting rather than one monolith.

When it pays — and how it fails

Debate is expensive: the cost is roughly N agents × R rounds of LLM calls, so reserve it for hard reasoning, math, and factuality questions where a wrong answer is costly — not for everything. Keep the final step cheap (a majority vote or a small judge model rather than another full debate round).

The characteristic failure is conformity: agents anchor on one confident-sounding answer and converge on it even when it’s wrong — a debate can manufacture a worse consensus than its best lone member. The antidote is diversity: different models, prompts, temperatures, or assigned roles, so the agents don’t share the same blind spot. Diversity is what made voting work, and it is what keeps debate honest.

In one breath

• A single agent that reasons wrong has no one to catch it; using several agents and combining them fixes that — by voting or by debating.
• Voting (aggregate independent answers) lifts accuracy via the wisdom of crowds: if agents beat chance with independent errors, the majority is right more often (5 agents at 60% → 68%, 15 → 79%).
• The lift depends on independence — same model + same prompt makes errors correlate and the vote barely helps.
• Debate has agents read each other and revise over rounds, breaking correlated errors a blind vote can’t — the idea behind multiagent debate and Minsky’s Society of Mind.
• Debate costs N agents × R rounds; its failure mode is conformity (a confident wrong answer wins), countered by diversity of models/prompts/roles.

Quick check

Next

Debate is cross-agent critique; the single-agent analogue is reflection (an agent critiquing its own output). Picking a final answer with a model is LLM-as-judge evaluation, and the topologies these agents run on are in multi-agent orchestration.