Tool calling (function calling)

LLMs can’t reliably do math. They don’t know today’s date. They have no idea what’s in your database. The fix isn’t a better prompt — it’s giving them tools. Tool calling is how the model says “I want to run this function with these arguments”, you run it, and feed the result back.

The contract

Tool calling is just structured output with a loop around it. The model never runs code — it emits a structured JSON object naming the tool and its arguments; your code executes the actual function and feeds the result back. The schema describes what tools exist; the model decides whether to use one and with what arguments; you execute and return the result; the model continues.

1. You:    "Here are tools: [get_weather, search_db]. User asks: '...' "
2. Model:  "Call get_weather(city='Tokyo')."           — structured output
3. You:    Run the function, get back {"temp_c": 18}.
4. You:    Append the result to the conversation, call the model again.
5. Model:  "The weather in Tokyo is 18 C."             — final answer

Every agent — LangGraph, LangChain, MS Agent Framework, raw API — is just this loop with bookkeeping.

Naming across providers

Same concept, different vocabulary. The code in this lesson uses Anthropic’s Messages API shape because it’s the cleanest illustration of the loop, but the other providers wrap the same concepts:

Provider	Term	API field
Anthropic	tool use	tools=[{name, description, input_schema}]
OpenAI	function calling / tools	tools=[{type: "function", function: {...}}]
Google Gemini	function calling	tools=[FunctionDeclaration(...)]
MCP (open standard)	tools	server-side @tool decorator

The contract is identical: JSON Schema in, structured tool call out, execution result back in. Once you’ve done a tool loop in one provider’s SDK, the others read fluently.

A weather tool, end to end

# A complete tool-calling loop with mocked API responses.
# In production, replace mock_call_model with: client.messages.create(...)

import json

# 1. Define the tool schema (this is what the API receives)
tools = [{
    "name": "get_weather",
    "description": "Get the current weather for a city",
    "input_schema": {
        "type": "object",
        "properties": {
            "city": {"type": "string", "description": "City name, e.g. 'Tokyo'"},
            "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
        },
        "required": ["city"],
    },
}]

# 2. The actual function the LLM is asking us to call
def get_weather(city: str, unit: str = "celsius"):
    # Pretend we hit a weather API
    fake_db = {"Tokyo": 18, "Lagos": 32, "Reykjavik": 4}
    temp = fake_db.get(city, 20)
    if unit == "fahrenheit":
        temp = temp * 9 / 5 + 32
    return {"city": city, "temperature": temp, "unit": unit}

# 3. Mock the model's responses for two turns of the loop
def mock_call_model(messages, turn):
    if turn == 0:
        # First turn: model decides to call the tool
        return {
            "stop_reason": "tool_use",
            "content": [{
                "type": "tool_use",
                "id": "toolu_01",
                "name": "get_weather",
                "input": {"city": "Tokyo", "unit": "celsius"},
            }],
        }
    else:
        # Second turn: model uses tool result to answer
        return {
            "stop_reason": "end_turn",
            "content": [{"type": "text",
                         "text": "It's currently 18°C in Tokyo."}],
        }

# 4. The loop
messages = [{"role": "user", "content": "What's the weather in Tokyo?"}]

for turn in range(5):  # always cap the loop
    response = mock_call_model(messages, turn)

    if response["stop_reason"] == "end_turn":
        print("FINAL:", response["content"][0]["text"])
        break

    # Find the tool call
    for block in response["content"]:
        if block["type"] == "tool_use":
            print(f"model wants: {block['name']}({block['input']})")
            result = get_weather(**block["input"])
            print(f"executed -> {result}")

            # Append the tool call AND the result to the conversation
            messages.append({"role": "assistant", "content": response["content"]})
            messages.append({"role": "user", "content": [{
                "type": "tool_result",
                "tool_use_id": block["id"],
                "content": json.dumps(result),
            }]})

model wants: get_weather({'city': 'Tokyo', 'unit': 'celsius'})
executed -> {'city': 'Tokyo', 'temperature': 18, 'unit': 'celsius'}
FINAL: It's currently 18°C in Tokyo.

Trace the three printed lines against the messages. The conversation grows: user question → assistant tool call → user tool result → assistant final answer. The model sees its own get_weather call in context on the second turn, which is why it can answer “18°C in Tokyo” without you ever passing the number back in words — it reads it from the tool_result block.

Controlling tool use with tool_choice

Anthropic’s Messages API (and similar in other SDKs) takes a tool_choice parameter that lets you nudge the model:

• {"type": "auto"} (default) — model decides whether to call a tool.
• {"type": "any"} — model must call one of the provided tools.
• {"type": "tool", "name": "..."} — force a specific tool. This is the canonical way to do structured outputs against Claude (define a tool whose input_schema is your output schema, then force-call it).
• {"type": "none"} — block tool use; reply with text only.

Default to auto. Reach for the others when you need to override.

Parallel tool calls

Modern APIs let the model request multiple tools in one turn. If the user asks “weather in Tokyo and Lagos”, the model can emit two tool_use blocks. You execute both (often in parallel) and return both results in a single user message (keyed by tool_use_id).

# Pseudocode
import asyncio
tool_calls = [block for block in response["content"]
              if block["type"] == "tool_use"]
results = await asyncio.gather(*[run_tool(tc) for tc in tool_calls])

This is a real latency win. Don’t process tool calls sequentially if the API gave you parallel ones.

What can go wrong

• Hallucinated tools — the model calls a tool you didn’t declare. Modern APIs prevent this with constrained generation, but old SDKs or local models still do it. Always validate the tool name exists.
• Wrong arguments — the model passes a string where you wanted an int. JSON Schema validation should catch this; surface the error back to the model as a tool result and it will usually self-correct.
• Tool errors — your function raises. Catch it, return the error as the tool result ({"error": "city not found"}), let the model decide what to do.
• No tool called when one should be — model decides to answer from training data instead of using the tool. Sharpen the tool’s description and add an explicit instruction: “Always call get_weather for any weather question.”

Designing good tools

A few rules from production:

• One job per tool. search_orders(customer_id) and get_order_details(order_id) beats one do_order_stuff(action, ...).
• Descriptive names and field docs. The model reads the schema. Treat tool descriptions like you’d treat docstrings for an underspec’d intern.
• Return structured results, not free text. A tool returning {"status": "ok", "data": {...}} is easier for the model to use than a sentence.
• Make idempotent reads obvious; gate writes. Tools that change state need a confirmation pattern — let the model propose, then a human or rule approves. Don’t give a model unrestricted DELETE.

In one breath

• A tool call is just structured output with a loop — the model emits JSON naming a tool and arguments; your code runs the function and feeds the result back.
• The message pattern: user question → assistant tool_use → user tool_result → assistant answer; the model reads its own call back from context.
• tool_choice steers it — auto (decide), any (must call one), a named tool (force it, the structured-output trick), or none.
• Always cap the loop (5-10 iterations), run parallel tool calls concurrently, and return errors as a tool_result so the model can self-correct.
• Tool calls are untrusted input — gate every destructive action behind a human or rule; never wire an LLM straight to DELETE.

Quick check

Quick check

0/3

Q1What does the LLM actually return when it 'calls a tool'?

It directly executes the functionA structured output (JSON) saying which tool to call and with what arguments — your code runs the toolA natural-language description that you parse with regexAn HTTP request

Check answer

Q2Why cap the tool-calling loop with a max_iterations?

API providers require itModels sometimes get stuck calling tools in loops; the cap prevents runaway costCaching only works with capped loopsTools fail after 10 calls

Check answer

Q3If a tool raises an exception in production, what should you do?

Let the exception propagate and crashHide the error from the modelCatch it and return the error as a structured tool result so the model can reactRetry the tool until it succeeds

Check answer

Next

The final lesson in this series is RAG basics — the pattern for making the model answer based on your data, not just its training.