What is tool use or function calling in LLMs, and how do you design good tools for an agent?

How to think about it

Tool use, also called function calling, lets a large language model emit a structured request for an external function; the application then validates and executes that request and gives the result back to the model. The model does not run the function itself. A reliable design makes the right tool easy to choose, the arguments hard to misunderstand, and dangerous actions impossible without an independent policy check.

Why function calling works

An LLM normally produces text. If a user asks, “Where is order A-1842?”, the model can write a plausible answer, but it cannot inspect the company’s order database unless the application gives it a way to do so.

The application supplies a tool definition alongside the conversation. That definition usually contains a name, a description, and an input schema: a machine-readable description of the allowed arguments and their types. The model sees those definitions as part of its decision context. It may then return a special tool-call message rather than ordinary prose.

The runtime owns the loop:

1. The user asks a question.
2. The model chooses a tool and emits arguments.
3. The runtime validates the arguments, checks authorization, and calls the real function.
4. The runtime sends the tool result back to the model.
5. The model answers the user or requests another tool.

The exact message format differs between model providers, but that division of responsibility does not. The model proposes an action. Your application decides whether the action is valid and allowed.

That distinction matters because a model can choose the wrong tool, omit a required argument, invent an identifier, call a tool twice, or claim success after a failed call. Function calling gives the output a structure. It does not give the model judgment, permission, or a guarantee of correctness.

A concrete example: refunding a customer

Suppose a support agent handles this request:

“I was charged twice for order A-1842. Please refund the duplicate $79 charge.”

The agent has two tools:

• get_order, which reads order and payment information.
• issue_refund, which creates a financial side effect.

A deliberately narrow refund tool might be described with a schema like this:

{
  "name": "issue_refund",
  "description": "Refund a captured duplicate payment for an order. Use only when the customer explicitly requests a refund and the order record confirms the duplicate charge.",
  "input_schema": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "description": "The exact order identifier, such as A-1842."
      },
      "amount_cents": {
        "type": "integer",
        "minimum": 1,
        "description": "Refund amount in whole US cents."
      },
      "currency": {
        "type": "string",
        "enum": ["USD"]
      },
      "idempotency_key": {
        "type": "string",
        "description": "Stable key preventing the same refund from being submitted twice."
      }
    },
    "required": ["order_id", "amount_cents", "currency", "idempotency_key"],
    "additionalProperties": false
  }
}

The model might first request:

{
  "tool_name": "get_order",
  "arguments": {
    "order_id": "A-1842"
  }
}

The real service returns that the order was delivered eight days ago, contains two captured payments, and has one duplicate payment worth 7,900 cents. The model can then propose:

{
  "tool_name": "issue_refund",
  "arguments": {
    "order_id": "A-1842",
    "amount_cents": 7900,
    "currency": "USD",
    "idempotency_key": "refund-A-1842-duplicate-7900"
  }
}

The runtime should not blindly execute this request. It should verify that the logged-in support user can issue refunds, that the order belongs to the relevant customer, that 7,900 cents is a refundable duplicate payment, and that this idempotency key has not already succeeded.

Only after those checks does it call the payment service. If the payment service returns an authoritative success result, the model can tell the customer that the $79 refund was submitted. If the service says the payment is still pending, the model must say that instead. The model’s confidence is not a payment receipt.

How I design good tools

Make the tool’s purpose obvious. Names such as get_order and issue_refund are easier to distinguish than customer_action or process_request. The description should say when to use the tool, what it changes, and when not to use it. “Refund a duplicate captured payment after verification” is much more useful than “Handles refunds.”

Descriptions are part of the model’s decision context, not decoration. Two tools with overlapping names and vague descriptions create selection errors. A description can guide the model, but it must never be treated as an authorization mechanism.

Use small, explicit inputs. Prefer a required amount_cents integer over a free-form string such as "refund": "$79, probably the duplicate one". Use enumerations for finite choices, state units in the field description, reject unknown fields, and make optional fields genuinely optional. If the runtime already knows the authenticated customer, do not ask the model to provide a customer_id that could be swapped for someone else’s.

Keep each tool narrow. A tool that searches orders, changes addresses, issues refunds, and sends emails may reduce the number of definitions, but it creates a large and ambiguous action surface. Narrow tools make both testing and authorization easier. In the refund example, reading an order and issuing a refund should remain separate because reading is reversible and issuing a refund is not.

Put security outside the model. Authenticate the user and enforce permissions in the runtime or service layer. Validate resource ownership, amount limits, fraud rules, and approval requirements there as well. A prompt saying “never refund more than $500” is useful guidance but is not a control.

For irreversible or expensive actions, require an explicit confirmation or human approval. A model may infer that “Can I get my money back?” means “issue the refund now,” but those are different requests. The distinction should be enforced by application state, not merely by careful wording.

Design for retries. Networks fail after a server receives a request but before the client receives the response. The runtime may retry, and the model may produce the same call again. A write tool should accept an idempotency key and make repeated requests safe. Without one, a timeout can turn a single intended refund into two refunds.

Return compact, truthful results. Return the fields the model needs to take the next step: status, identifiers, relevant amounts, and a human-readable explanation. Do not dump an entire database row or a stack trace into the context. Tool output should also be treated as untrusted data. A document returned by a search tool can contain text that tries to manipulate the model; it is not a new system instruction.

Errors should explain recovery. For example:

{
  "ok": false,
  "error": {
    "code": "REFUND_WINDOW_CLOSED",
    "message": "Order A-1842 was delivered 45 days ago; refunds are allowed within 30 days.",
    "retryable": false,
    "suggested_action": "Offer store credit or escalate to a human."
  }
}

A boolean false tells the model almost nothing. A typed error tells it whether to retry, ask the user for information, or stop.

The senior-level trade-off

More tools do not automatically make an agent more capable. Tool definitions consume context, and similar tools compete for the model’s attention. An agent with 40 overlapping search and update tools may perform worse than one with eight carefully separated tools. Grouping operations behind a router can help, but the router becomes another component that needs evaluation.

The right amount of autonomy depends on risk, reversibility, latency, and cost. A read-only order lookup can often run automatically. A bank transfer should normally require stronger checks and approval. For a deterministic workflow that always calls the same three services in the same order, ordinary application code is usually cheaper, faster, and easier to audit. An LLM is useful when interpreting messy language or choosing among genuinely variable next steps—not because every API call deserves a tiny probabilistic manager.

A failure mode to recognize

The first symptom is often a log full of repeated calls such as issue_refund followed by timeout errors, or validation errors saying the amount is missing. The underlying causes are usually ambiguous schemas, side-effecting tools without idempotency, or a runtime that lets the model retry forever.

Set a maximum number of tool turns, validate every argument before execution, log the model’s requested call and the service’s actual result, and make retries depend on the error’s retryable status. Also ensure the final answer is generated from the returned result. Otherwise the agent may say “Your refund is complete” even though the payment service rejected it.

What they’ll ask next

How is tool use different from retrieval-augmented generation?
Retrieval supplies information to help answer a question. A tool performs an operation, such as querying an account, sending an email, or changing a record. A system can use both: retrieve the refund policy, then call the refund tool, with the runtime enforcing the policy.

How do you stop prompt injection from causing a dangerous tool call?
Treat model output and tool output as untrusted. Use allowlists, server-side authorization, input validation, scoped credentials, confirmation for high-impact actions, and isolation for code or browser tools. Never let text from a retrieved document grant permission.

What do you evaluate?
Measure tool-selection accuracy, argument validity, unnecessary calls, recovery from errors, latency, cost, and side-effect safety. Include adversarial cases: missing order IDs, conflicting instructions, duplicate requests, expired permissions, and a timeout after the external service has already completed the action.

Say this in the interview

“Function calling is a controlled application loop: the model proposes a typed tool call, the runtime validates and authorizes it, executes the real function, and returns the result; I design tools to be narrow, explicit, least-privilege, idempotent, and honest about errors.”