Durable execution for agents
An agent run isn't a request-response — it's a long workflow of LLM calls, tool calls, waits, and approvals. If the process crashes mid-run you lose all progress, and you can't blindly re-run because some steps already had side effects. Durable execution makes a workflow survive crashes and resume exactly where it left off.
What you'll learn
- Why long agent workflows need crash-proof state, not just retries
- Workflows vs activities, and the durable event log
- How replay reconstructs state without re-running completed side effects
- Durable timers, the frameworks (Temporal, Restate), and when to reach for them
Before you start
A chat completion is over in seconds; an agent run is not. It’s a long workflow — a sequence of LLM calls, tool invocations, waits, and human approvals that can stretch from minutes to days. That length creates a reliability problem ordinary request-response code never faces: if the process crashes partway through (a deploy, an OOM kill, a network blip), you lose all the work so far. And you can’t just re-run it from the top, because some steps already had side effects — you must not re-charge the card or re-send the email. Durable execution is the runtime that solves exactly this.
Workflows, activities, and the event log
A durable-execution engine splits your code into two kinds of thing:
- Workflow — the deterministic orchestration logic (the plan: do this, then that, then wait for approval). It must be replayable, so it can’t contain randomness or direct side effects.
- Activities — the side-effectful steps: an LLM call, a tool, an API request, a payment. The engine persists each activity’s result to a durable event log as it completes.
That log is the whole trick. When the process dies and restarts, the engine replays the workflow code from the beginning — but instead of re-executing activities it has already recorded, it feeds back their stored results, deterministically rebuilding the state, and then continues from the point of failure. Completed side effects are never repeated:
history = {} # the durable event log: step -> recorded result
def activity(step, fn):
if step in history: # replay: return the recorded result, don't re-run
print(f" {step}: replayed (cached) -> {history[step]}")
return history[step]
result = fn() # first run: execute the side effect
history[step] = result
print(f" {step}: EXECUTED -> {result}")
return result
print("attempt 1 (crashes after charge):")
activity("reserve", lambda: "ok")
activity("charge", lambda: "$42 charged")
# ... process crashes here ...
print("attempt 2 (resume from durable history):")
activity("reserve", lambda: "ok")
activity("charge", lambda: "$42 charged") # replayed — the card is NOT charged again
activity("ship", lambda: "shipped") # the new step, executed once
attempt 1 (crashes after charge):
reserve: EXECUTED -> ok
charge: EXECUTED -> $42 charged
attempt 2 (resume from durable history):
reserve: replayed (cached) -> ok
charge: replayed (cached) -> $42 charged
ship: EXECUTED -> shipped
On the second attempt reserve and charge are served from the log — the payment
isn’t repeated — and only the new ship step runs. The workflow continues as if the
crash never happened.
Durable timers and the long wait
The same machinery enables something agents need badly: durable waits. An agent can “sleep” for hours or days — waiting for a human approval, a webhook, a scheduled time — and the engine survives restarts, deploys, and machine failures across that whole window. There’s no process sitting in memory holding a timer; the wait is just another entry in the durable state, resumed when the time comes.
In one breath
- An agent run is a long workflow (LLM calls, tools, waits, approvals); a crash mid-run loses progress, and you can’t blindly re-run because of side effects (don’t re-charge, don’t re-send).
- Durable execution splits code into a deterministic workflow and side-effectful activities, persisting each activity’s result to a durable event log.
- On crash + restart it replays the workflow, feeding back recorded activity results instead of re-running them — reconstructing state and resuming at the failure point, with completed side effects never repeated (the demo: charge isn’t repeated).
- Durable timers let an agent wait hours or days (for approval/webhook) across restarts, since the wait lives in durable state.
- Use Temporal / Restate / DBOS / Step Functions (or LangGraph checkpointing) when an agent is long-running, stateful, waits on humans, or takes irreversible actions.
Quick check
Quick check
Next
Durable execution keeps long agent runs reliable; LangGraph persistence is the lighter in-framework version, and observability is how you watch those runs. For bounding what an autonomous agent is allowed to do, see agent safety controls.