What are reasoning models, and what is test-time compute?

How to think about it

Reasoning models are language models optimized to spend extra computation on intermediate reasoning before producing an answer. Test-time compute is the broader strategy of spending more computation after training, during each request, through longer reasoning, multiple candidate solutions, verification, search, or tool use.

Why the distinction matters

A conventional autoregressive language model generates an answer one token at a time. An autoregressive model predicts the next token from the tokens already in its context. For an easy question, it may go directly from the prompt to the answer. For a difficult question, that direct path can fail because it must plan, calculate, and obey several constraints at once.

A reasoning model gets an opportunity to use extra intermediate steps as a scratchpad. It might decompose a problem, calculate partial results, notice a contradiction, and try another route before returning the final answer. Those steps are not free: each generated reasoning token requires additional inference work.

The training recipe varies. Common approaches use worked reasoning examples, reinforcement learning from outcome rewards, process supervision, or a combination. Outcome supervision rewards a correct final answer. Process supervision also evaluates intermediate steps. The model learns that spending computation on useful intermediate work leads to better outcomes, rather than merely learning to sound thoughtful.

That last distinction matters. A long explanation is not automatically reasoning. A model can produce ten paragraphs of confident nonsense. The useful property is not length by itself; it is the ability to use additional computation to improve the probability of a correct, constraint-satisfying answer.

Common misconception: “Test-time” does not mean evaluation on a held-out test set. It means the time when the trained model is answering a request, also called inference time. The model weights stay fixed; the system spends more compute per question.

The mechanism

Suppose a model receives a scheduling problem with six jobs, precedence constraints, and a two-hour deadline. A direct response may commit to the first plausible ordering. A reasoning model can write down the constraints, test an ordering, discover that job D now starts after the deadline, and revise the schedule.

The model has not gained new facts merely by thinking longer. It has gained more opportunities to transform the facts in the prompt into a valid answer. This is why reasoning models tend to help most on problems involving mathematics, code, planning, formal constraints, and multi-step decisions.

There are several ways to spend that extra compute:

Strategy	What it does	Strongest use case
Longer reasoning trace	Gives one solution more intermediate steps	Complex but coherent problems
Multiple samples	Generates several independent attempts	Problems with a checkable answer
Verification	Scores or tests candidate answers	Code, arithmetic, proofs, structured output
Search	Explores several possible next steps	Planning, games, combinatorial problems
Tool use	Calls a calculator, database, retriever, or interpreter	Problems needing current facts or exact computation

“Chain of thought” usually refers to the intermediate reasoning text. A reasoning model may produce such text internally, but the visible explanation is not necessarily a faithful transcript of the computation that caused the answer. In production, I would evaluate the answer with independent checks rather than treating a persuasive rationale as proof.

A concrete example

Consider this invoice:

• Original price: $240
• Discount: 15%
• Tax: 8% applied after the discount

The correct calculation is:

1. Discounted subtotal: $240 × 0.85 = $204
2. Final total: $204 × 1.08 = $220.32

A direct model can make either of two common mistakes. It can apply the tax to the original $240, or it can stop after calculating the discounted subtotal. A reasoning model has room to represent the order of operations explicitly.

The arithmetic is independently checkable:

from decimal import Decimal

price = Decimal("240")
subtotal = price * Decimal("0.85")
total = subtotal * Decimal("1.08")

print(subtotal.quantize(Decimal("0.01")))
print(total.quantize(Decimal("0.01")))

Output:

204.00
220.32

Now imagine a test-time controller asks for five candidate solutions and receives:

$220.32, $220.80, $220.32, $204.00, $220.32

Majority voting selects $220.32, with three of five candidates agreeing. A calculator verifier is better still: it can evaluate the discount and tax directly instead of trusting the model to judge its own arithmetic.

This is test-time compute. We did not retrain the model or increase its parameter count. We used more generation and an external check for one request.

The senior nuance: more compute is not always better

The benefit depends on the task and on the quality of the extra work.

Multiple samples help only when their errors are not perfectly correlated. If five attempts all inherit the same false assumption, voting simply produces the same wrong answer five times. Under an idealized assumption that each attempt is independent and has accuracy 0.70, majority voting across five attempts has accuracy 0.83692. Real model samples are usually correlated, so the improvement is often smaller.

A verifier can also be weak. Asking the same model, “Is this answer correct?” may reward fluent reasoning rather than truth. For code, run tests. For arithmetic, use a calculator. For a database question, execute the query against a known fixture. For open-ended writing, use a rubric or human review, while accepting that there may be no objectively correct winner.

Test-time compute is especially valuable when correctness is verifiable. It is less magical for subjective copywriting, ambiguous policy questions, or questions requiring facts the model does not have. Ten minutes of internal reasoning cannot recover a sales figure that was never in the model and was not retrieved from a current source.

There is also a practical cost curve. If one candidate uses about 100 reasoning tokens, five candidates use roughly 500 candidate tokens before verification. Parallel generation can reduce wall-clock latency, but it still increases token cost, concurrency, and peak load. A system that improves offline accuracy by 4 percentage points but pushes p95 latency from 1.5 seconds to 9 seconds may be a regression for an interactive product.

Reasoning can even hurt easy tasks. Extra steps create more opportunities for arithmetic slips, instruction drift, and invented premises. Good systems therefore route requests adaptively: use a small budget for a simple lookup, and reserve deeper reasoning or search for cases that justify it.

What I would build in production

I would start with a task-specific success criterion. “The answer sounds better” is not an evaluation.

Then I would set a compute budget and use a staged policy:

1. Try a low-cost response.
2. Detect whether the task is complex or whether the answer fails a cheap check.
3. Allocate more reasoning, generate additional candidates, or call a tool.
4. Select the result with an independent verifier where possible.
5. Log answer quality, reasoning tokens, total tokens, latency, verifier outcomes, and escalation rate.

For a coding assistant, that might mean one draft followed by execution of unit tests. For a financial calculation, it might mean structured extraction followed by deterministic arithmetic. For a planning task, it might mean several candidate plans followed by constraint checking.

The controller should optimize expected value, not raw accuracy. A difficult request may justify 8 candidates and a verifier. A simple request may not justify even 2. The right question is whether the improvement pays for its latency and compute.

A failure mode to watch for

The first symptom is often a rising token and latency dashboard with no improvement in evaluation accuracy. If a system moves from one sample to eight and the model keeps returning the same wrong answer, the likely causes are correlated errors, a weak verifier, or a reasoning budget aimed at the wrong part of the problem.

Another symptom is a polished explanation attached to an invalid result. That usually means the system is rewarding verbal confidence instead of checking the actual output. Add an executable or deterministic check before adding more tokens.

What they’ll ask next

Is chain of thought the same thing as a reasoning model?

No. Any sufficiently capable language model can be prompted to show intermediate steps. A reasoning model is trained or optimized to use additional inference computation more reliably, often with specialized post-training. Visible chain-of-thought text is an interface choice, not a reliable definition.

How is test-time compute different from making the model larger?

Making the model larger changes its learned parameters and usually increases the cost of every request. Test-time compute leaves the parameters unchanged and spends extra work selectively on particular requests. They are complementary: a larger model may solve more problems directly, while extra inference compute may improve difficult problems without serving the largest model for every user.

Does more test-time compute always improve accuracy?

No. It helps when extra attempts contain useful diversity and when a verifier can identify the right answer. It has diminishing returns, adds cost and latency, and cannot fix missing information or a shared false premise. I would measure the quality-versus-cost curve on the actual workload rather than assume that a longer reasoning trace is better.

Say this in the interview: Reasoning models spend extra inference computation on intermediate problem-solving, while test-time compute is the broader practice of buying better answers with more work per request, using longer reasoning, multiple candidates, verification, search, or tools.