Your team calls every LLM workflow an agent. How do you distinguish a genuinely agentic system from a deterministic chain, and when would you deliberately choose the chain?

How to think about it

A deterministic chain follows a predesigned sequence of steps, while a genuinely agentic system decides its next action at runtime from its goal, current state, and observations, with the ability to replan or stop. I would deliberately choose the chain when the task is bounded and predictable, especially when side effects, latency, cost, and auditability matter more than open-ended adaptation.

Why the distinction matters

The interviewer is probing control flow, not whether a language model can call a tool.

A chain has a known route. It might contain an LLM at every step, and it might even have branches. But the application designer has specified those steps and branches ahead of time.

An agent has a goal and a set of possible actions. At runtime, it uses the current state and the result of its previous action to choose what to do next. The route is not fully known in advance.

The basic agent loop looks like this:

1. Read the goal and current state.
2. Choose an action.
3. Execute a tool or produce an answer.
4. Observe the result.
5. Update state.
6. Continue, replan, ask for help, or stop.

The important word is choose. The system is not merely filling in text inside a fixed pipeline. Its decision changes the next operation.

That does not mean every system with an if statement is an agent. A workflow with four predefined branches is still a deterministic workflow if the application has already decided what each branch means. “Deterministic” here usually describes the control flow, not the model output. A fixed chain can still produce slightly different wording because an LLM is probabilistic.

Question	Deterministic chain	Agentic system
Who determines the next step?	The workflow definition	A runtime decision-maker, within constraints
Can the route change after an observation?	Only through predefined branches	Yes, including new plans or alternate tools
Typical stopping rule	The final configured step	Goal reached, failure, approval, or budget exhausted
Main strength	Predictability	Adaptation to unknown situations
Main risk	Brittle handling of variation	Unbounded cost, actions, or loops

A useful test is this: remove the model’s freedom to select the next action. If the same steps still run in the same order, you have a chain with LLM components. If the system must inspect the situation and decide which available action is appropriate, you have something meaningfully agentic.

A concrete example: the 3:07 a.m. duplicate charge

Imagine a support ticket arriving at 3:07 a.m.:

“I was charged twice for the same order. The amount is $42.”

A deterministic workflow might always do this:

1. Extract the customer and order ID.
2. Retrieve the order.
3. Retrieve payment records.
4. Check the duplicate-charge policy.
5. Issue a refund if the policy allows it.
6. Send a confirmation email.

That is a chain even if an LLM performs steps one, four, and six. The route is designed beforehand. If the order ID is missing, the workflow can follow a predefined “ask the customer” branch. If the payment service times out, it can retry twice and then create a human-review ticket.

Now give an agent the goal “resolve the customer’s duplicate-charge request” and tools such as customer lookup, order search, payment inspection, policy lookup, refund creation, and email drafting. Those tool names are illustrative; the point is the shape of the system, not a vendor API.

The agent might search by email because the ticket contains no order ID. It might discover two payment authorizations but only one captured payment. That observation changes the next action: explaining the temporary authorization may be correct, while issuing a refund would be wrong. Or it might find two captured payments, check the policy, create a refund, and draft a message.

The agent has selected different actions because the evidence changed. It has not merely filled different values into the same six boxes.

A production version would still impose hard limits. For example, the system could allow at most eight tool calls or 90 seconds of runtime, permit refunds only below a configured amount without approval, and require the payment service to enforce authorization independently of the model. The agent may propose a refund, but the model should not be the final security boundary.

When I would choose the chain

I would choose the chain for the duplicate-charge workflow if the business rules were stable and the actions had financial consequences.

The reason is not that agents are “bad.” It is that a chain makes important properties easier to guarantee:

• Auditability: every ticket follows a known sequence, so an investigator can see where it failed.
• Testing: each step has a clear input and expected output.
• Latency and cost: the workflow can budget a known number of model and service calls.
• Safety: refund creation can occur only after explicit checks in the application.
• Operations: retries, timeouts, idempotency, and human escalation can be designed per step.

A chain is especially sensible for document processing, invoice extraction, onboarding checks, routine classification, and regulated decisions where the acceptable route is known. If the task is “extract these twelve fields, validate them, and store the record,” giving a model permission to invent a new plan is mostly adding risk.

The chain becomes less attractive when the input and route are genuinely unpredictable. Consider a production incident assistant. One incident may require checking logs, another a recent deployment, another a feature flag, and another a dependency status page. The useful order depends on what each observation reveals. A bounded agent can choose among those investigations, revise its hypothesis, and stop when it has enough evidence for a human operator.

Even there, I would usually build a hybrid: deterministic outer controls for authentication, budgets, approvals, and escalation; an agentic inner loop for selecting investigative actions. “Agentic” does not mean “the model owns the whole system.” It often means one carefully fenced part of the system has runtime choice.

The senior nuance: agency is a continuum

The textbook distinction is useful, but real systems sit on a spectrum.

A chain can have dynamic data-dependent branches and still be highly adaptive. An agent can be tightly constrained to one of three tools and one retry. The question is not whether the system has a loop or uses function calling. The question is how much of the next action is specified before execution and how much is selected from runtime evidence.

Tool use alone does not make an agent. A workflow that always calls search, then always calls a database, then always calls an email service is still a chain. Conversely, an agent does not need dozens of tools. A system that chooses between “ask a clarifying question” and “run a lookup” based on the current state already has runtime action selection.

There is also a practical failure mode. An agent may repeatedly call the same search tool because each result fails to satisfy its internal notion of progress. The first symptom is usually visible in the trace: repeated tool calls, rising token usage, and eventual budget exhaustion without a useful answer. A chain tends to fail differently: a missing order ID may send every ticket into the same broken lookup step.

For an agent, I would add an allowlist of tools, typed arguments, server-side authorization, time and token budgets, loop detection, structured traces, and explicit approval for consequential actions. For a chain, I would test every branch, make retries safe through idempotency, and monitor where inputs fail to match the expected schema.

The strongest answer is therefore not “agents are smarter” or “chains are safer.” It is: use runtime choice where the environment is uncertain, and use predetermined control flow where correctness and control are the product.

What they’ll ask next

“Is a loop that calls tools automatically an agent?”

Not necessarily. A loop that repeats a fixed sequence is still a chain. I would look for runtime action selection, feedback from the environment, a goal or stopping condition, and the ability to choose an alternate path. I would also ask what prevents the loop from running forever.

“How would you evaluate the two designs?”

For a chain, I would evaluate each step and branch: extraction accuracy, policy-check accuracy, tool error handling, end-to-end success, latency, and cost. For an agent, I would additionally measure tool-selection quality, unnecessary calls, recovery from bad observations, unsafe-action rate, termination behavior, and the full trajectory rather than only the final answer.

“Would you ever replace an agent with a chain?”

Yes. If traces show that the agent takes the same six actions in nearly every successful case, I would consider turning those actions into a chain and keeping only the genuinely variable decision as a bounded agentic step. That usually improves predictability without giving up useful adaptation.

One line to say in the room

“I distinguish them by who controls the next action: a chain follows a route I designed, while an agent selects and revises its route from runtime observations; I choose the chain whenever the route is known and predictability is more valuable than flexibility.”