OpenTelemetry for GenAI

Observability is non-negotiable for agents — they’re non-deterministic, multi-step, and spread across model calls, tool invocations, and retrieval, so “it failed” is useless without a trace of where and why. The open standard for that is OpenTelemetry (OTel), and its GenAI semantic conventions fix the one thing that otherwise makes LLM observability a mess: a common vocabulary, so every tool records and reads telemetry the same way.

OpenTelemetry in one minute

OTel is a vendor-neutral standard for telemetry with a few core ideas:

• A trace is the full journey of one request.
• A span is a single unit of work within it (an LLM call, a tool call), and spans nest — child spans sit inside their parent, forming a tree.
• Attributes are key-value metadata on a span; metrics are aggregated measures.

Because it’s vendor-neutral, you instrument once and export to any backend — Jaeger, Langfuse, Phoenix, Datadog, Grafana. Switch tools without re-instrumenting.

The GenAI semantic conventions

A trace is only portable if everyone agrees on the names. The GenAI semantic conventions standardize the attributes and span structure for LLM and agent telemetry, so a “chat” span everywhere carries the same fields:

• gen_ai.system — the provider (e.g. anthropic)
• gen_ai.request.model — the model id
• gen_ai.operation.name — chat, embeddings, …
• gen_ai.usage.input_tokens / gen_ai.usage.output_tokens — token counts
• plus span structure for tool calls and multi-agent handoffs

With those agreed names, one agent run becomes a readable span tree — and any OTel-compatible tool can render it, compute latency per step, and total the tokens:

# Spans from one agent run, each carrying the standard gen_ai token attributes.
spans = [
    {"name": "chat",        "gen_ai.usage.input_tokens": 1200, "gen_ai.usage.output_tokens": 80},
    {"name": "tool.search", "gen_ai.usage.input_tokens": 0,    "gen_ai.usage.output_tokens": 0},
    {"name": "chat",        "gen_ai.usage.input_tokens": 1500, "gen_ai.usage.output_tokens": 200},
]
inp = sum(s["gen_ai.usage.input_tokens"] for s in spans)
out = sum(s["gen_ai.usage.output_tokens"] for s in spans)
print(f"trace total: {inp} input + {out} output tokens across {len(spans)} spans")

trace total: 2700 input + 280 output tokens across 3 spans

Because every model span reports tokens under the same attribute name, totalling cost or finding the most expensive step is a trivial aggregation — no per-vendor parsing.

In one breath

• Agents need observability (traces of where/why), and OpenTelemetry is the vendor-neutral standard: a trace is a request’s journey, spans are nested units of work, attributes are metadata, exported to any backend.
• The GenAI semantic conventions standardize the names — gen_ai.system, gen_ai.request.model, gen_ai.operation.name, gen_ai.usage.input_tokens/output_tokens — plus span structure for tool calls and multi-agent.
• A standard schema makes observability portable: instrument once, render in Langfuse / Phoenix / Datadog / Grafana without re-instrumenting.
• A full agent run becomes a span tree (LLM → tool → LLM); since every model span reports tokens the same way, totalling cost or finding the slow step is a simple aggregation (2700 + 280 tokens in the demo).
• It’s the standard underneath agent observability: the practice traces; the conventions make it tool-agnostic.

Quick check

Next

Standardized telemetry feeds agent observability (the practice), the cost view in cost & latency, and agent evaluation (traces are what you score).