LlamaIndex agents: FunctionAgent and ReActAgent

At 3 a.m., a customer asks:

“Is the Pro plan refundable after 20 days, and does it include SSO?”

A query engine is configured for a particular data source or retrieval task. It can answer questions over that scope, but it is not automatically a router across separate authorities. Here, the refund rule lives in a policy index and SSO lives in product documentation. One retrieval pass may find one and miss the other.

You could write routing code yourself:

1. Send refund questions to the policy index.
2. Send feature questions to the product index.
3. Combine the results.
4. Ask a language model to write the answer.

That works, but special cases multiply when users ask about invoices, regional terms, or contract exceptions.

A LlamaIndex agent is a model-driven loop that chooses tools, receives their results, and decides whether another action is needed. QueryEngineTool wraps a query engine as one of those tools.

That is how retrieval-augmented generation, or RAG, becomes agentic: retrieval is one action the agent can choose, not the whole program.

Your query engine becomes a tool

An index is LlamaIndex’s searchable representation of your data. A query engine is the runnable interface that retrieves relevant pieces and synthesizes a response. It might use vector search, keyword search, SQL, or a combination.

QueryEngineTool adds what an agent needs:

• a stable tool name;
• a description written for the model;
• an input schema, usually a text query;
• the query engine that performs the work.

The agent does not inspect your vector database. It sees the tool name and description in the model request, so the description is part of routing logic.

For the running example, create two tools:

from llama_index.core.tools import QueryEngineTool

docs_tool = QueryEngineTool.from_defaults(
    query_engine=docs_index.as_query_engine(),
    name="product_docs",
    description=(
        "Answers questions about product features, plan limits, "
        "and SSO availability. Does not define refunds or legal terms."
    ),
)

policy_tool = QueryEngineTool.from_defaults(
    query_engine=policy_index.as_query_engine(),
    name="policies",
    description=(
        "Answers questions about refunds, cancellations, SLAs, and terms. "
        "Does not describe product features."
    ),
)

The two engines might use the same vector-store technology, but their indexed documents and descriptions establish different authorities.

Follow one request through the loop

Suppose the user asks:

“Is the Pro plan refundable after 20 days, and does it include SSO?”

The model first receives the question and the two tool definitions, not the contents of either index. It chooses a call:

tool: policies
arguments: {"input": "Is the Pro plan refundable after 20 days?"}

With QueryEngineTool.from_defaults, the default structured argument is named input. A custom schema could call it query; using the wrong field can cause a missing-input or invalid-tool-call error.

LlamaIndex executes the call. The policy query engine retrieves relevant chunks, perhaps saying refunds are available within 30 days and describing annual-contract exceptions. It returns a query-engine response to the agent.

The model sees that result and notices the SSO part is unanswered. It calls the other tool:

tool: product_docs
arguments: {"input": "Does the Pro plan include SSO?"}

The product engine returns the feature documentation. The model now has evidence for both parts and writes the final response.

The arithmetic matters. Two serial tool calls require at least:

• one model request to choose the first action;
• one policy query;
• another model request to interpret that result;
• one product query;
• a final model request to combine the evidence.

That is three model turns around two tool calls. If each model request takes 0.8 seconds and each query-engine call takes 0.2 seconds, rough serial latency is 3 × 0.8 + 2 × 0.2 = 2.8 seconds. These are planning numbers, not a benchmark.

The agent is not “searching harder.” It is changing the sequence of operations based on intermediate results.

FunctionAgent and ReActAgent

The central difference between LlamaIndex’s common agent styles is how the model expresses its next action.

FunctionAgent: structured actions

Function calling, or tool calling, is a model API feature in which the model returns a structured tool name and arguments instead of ordinary prose.

A FunctionAgent uses that native interface. LlamaIndex can validate the call, execute the tool, and place its result back into the workflow. The provider’s protocol carries the boundary, so the model does not need to print an exact Action: format.

Use FunctionAgent when the model and endpoint support native tool calling. Structured calls are easier to validate, log, retry, and reject safely.

ReActAgent: actions written as text

ReAct, short for “reasoning and acting,” alternates between an intended action and an observation:

Thought: The refund rule belongs in the policy documents.
Action: policies
Action Input: Is the Pro plan refundable after 20 days?
Observation: Refunds are available within 30 days, subject to contract terms.

ReAct is useful when a model lacks native tool calling, and its visible action sequence can be easy to inspect. But text is a fragile wire protocol: the model may change a field name, add commentary, emit invalid JSON, or mention a tool without producing a parseable call.

The visible trace is an action trace, not a guarantee that private internal reasoning has been exposed. Log tool names, arguments, results, and decisions rather than treating a long “Thought” paragraph as an audit record.

Both agents are built on LlamaIndex Workflows: event-driven execution graphs whose steps receive events, perform work, and emit new events. In an agent, those steps are essentially “ask the model,” “run the requested tool,” and “decide whether the run is finished.”

You can stream events, add processing, and trace the run using the concepts in LlamaIndex Workflows. When the prebuilt loop no longer fits, a custom Workflow is often clearer than adding instructions to the model.

A small working agent

Using the two tools above, the agent and asynchronous entry point look like this:

import asyncio

from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.tools import QueryEngineTool

agent = FunctionAgent(
    tools=[docs_tool, policy_tool],
    llm=llm,
    system_prompt=(
        "Use the available tools for product and policy facts. "
        "If a question has two independent parts, check both sources. "
        "If the sources do not establish an answer, say so."
    ),
)


async def main():
    handler = agent.run(
        user_msg="Is the Pro plan refundable after 20 days, and does it include SSO?"
    )
    response = await handler
    print(response)


if __name__ == "__main__":
    asyncio.run(main())

llm must be an LLM integration that supports the function-calling behavior required by FunctionAgent. The prompt helps, but it is not a security boundary: enforce authorization in tool implementations, and treat retrieved text as data rather than instructions.

Tool descriptions are routing policy

Descriptions such as these are too vague:

product_docs: Product information.
policies: Company information.

Both tools now appear plausible for “Is Pro refundable?” A useful description says:

1. what subject the tool covers;
2. what questions should trigger it;
3. what it does not cover;
4. what authority or output to expect.

“Answers questions about refunds, cancellations, SLAs, and terms; does not describe product features” defines both positive and negative boundaries. Names matter too—policies is clearer than tool_2—but routing depends on the name, description, question, and conversation together.

Keep descriptions truthful as indexes change. If a policy index contains only US consumer terms, do not describe it as worldwide authority.

The production pattern

A useful design has four layers:

• Query layer: owns ingestion, chunking, retrieval, reranking, and synthesis. Test the query engine directly before adding an agent.
• Tool layer: gives each capability a narrow name, description, input contract, and authorization check. QueryEngineTool is for question-answering, not a replacement for write tools.
• Agent layer: chooses tools and composes results. It should not enforce permissions that the application can check deterministically.
• Workflow boundary: sets a maximum number of tool steps, a deadline, and cancellation. Repeated calls are budget use, not thoroughness.

Use a custom Workflow when the sequence is known. “Look up account, check eligibility, request approval, then issue a refund” is safer as structured control flow, with an agent handling only ambiguous lookup.

Failure modes you will actually see

The wrong index is called

Overlapping descriptions, vague names, and genuinely multi-domain questions can route to the wrong tool. Make boundaries and negative scope explicit, then evaluate routing separately from answer quality with labelled, ambiguous questions.

The agent answers without a tool

A model may rely on pretrained knowledge or classify a factual question as casual conversation. State when tools are required, then verify the trace. If grounding is mandatory, make “no supporting tool result” an explicit uncertainty or review outcome.

ReAct parsing breaks

Near-matches such as an explanation before Action: or an unknown tool name can cause parser errors. Prefer FunctionAgent; otherwise validate names and arguments and treat parser failure as recoverable. Never execute arbitrary text because it resembles a command.

The loop repeats or citations disappear

Repeated searches usually mean the result was unclear or retrieval returned poor context. Set step limits, log normalized requests, return concise results, and improve retrieval before increasing the budget.

If source nodes vanish between the query engine and final prose, preserve source metadata through the workflow and test citation correctness separately. Citing something is not the same as citing the passage supporting a claim.

The honest trade-off

Agents buy flexibility by adding probabilistic decisions. A direct query engine may take one model call and one retrieval operation; an agent may take several of each, including a wrong turn. That costs money, latency, and test surface.

Use an agent when the request requires routing, iteration, or tool choice. For a fixed question-answering endpoint over one corpus, call the query engine directly.

Start with a query engine. Add QueryEngineTool when you can name the decision the agent must make.

In one breath

• QueryEngineTool exposes a query engine with a name, description, and callable interface.
• The agent chooses a tool, receives its result, and may choose another before answering.
• FunctionAgent uses structured native calls; ReActAgent uses text actions and a parser.
• Both run as LlamaIndex Workflows and should be bounded, traced, and evaluated.
• Fixed or consequential sequences belong in a Workflow.

Quick check

Next

The agent can only retrieve what your index successfully ingested. LlamaParse helps with the messy PDFs that quietly poison otherwise sensible RAG systems.