Skill libraries & sleep-time compute

Most agents have amnesia about how to do things. Solve a task today, and tomorrow the same kind of task is approached from scratch — the same reasoning re-derived, the same tokens re-spent. Humans don’t work that way: we learn a skill once and reuse it forever, building a repertoire. A skill library gives an agent that ability — solve a task once, save the solution as a reusable skill, then retrieve and compose skills for future tasks. It is lifelong learning that happens in the agent’s memory, not in its weights.

Voyager: the canonical example

The clearest demonstration is Voyager, an agent that plays Minecraft open-endedly. Its loop:

• Generate a skill. When it solves a task, it writes the solution as an executable code function (e.g. craftStoneSword()), not a one-off chat reply.
• Store it. The skill goes into a library, indexed by an embedding of its description so it can be found later by meaning.
• Retrieve and compose. For a new task, it embeds the task, retrieves the most relevant skills, and composes them — building complex behaviors out of saved simpler ones.
• Curriculum + refinement. An automatic curriculum proposes progressively harder tasks, and when a skill fails, iterative prompting refines it using the error feedback (a cousin of reflection).

The result is an ever-expanding repertoire: Voyager kept acquiring new abilities and generalized to new worlds far better than agents without a persistent skill store. The idea generalizes well beyond games — a coding agent that saves a helper function once and reuses it, an RPA agent that banks a verified procedure. This is procedural memory (how to do things), distinct from the episodic/semantic memory in agent memory.

The payoff: solve once, reuse cheaply

Re-deriving a procedure costs a full reasoning round; retrieving a saved skill is nearly free. Across repeated task types, the library turns most work into cheap reuse:

gen_cost, reuse_cost = 100, 10        # cost to generate a new skill vs reuse one
tasks = ["nav", "mine", "craft", "nav", "craft", "mine", "nav", "craft", "mine", "nav"]

without = len(tasks) * gen_cost        # no library: derive every task from scratch
seen, with_lib = set(), 0
for t in tasks:
    if t in seen:
        with_lib += reuse_cost          # retrieve & reuse the saved skill
    else:
        with_lib += gen_cost            # first of its kind: generate and save
        seen.add(t)

print(f"without library: {without}")
print(f"with library   : {with_lib}  ({len(seen)} skills generated, {len(tasks) - len(seen)} reused)")
print(f"reduction: {without / with_lib:.1f}x")

without library: 1000
with library   : 370  (3 skills generated, 7 reused)
reduction: 2.7x

Only the first task of each kind pays the full generation cost; the seven repeats reuse saved skills cheaply. The more an agent runs in a domain, the more its library covers and the cheaper it gets — capability that compounds, with no retraining.

Sleep-time compute: work off the query path

A complementary idea: not all the agent’s thinking has to happen when a query arrives. Sleep-time compute uses the agent’s idle time to do work in advance — digest and re-summarize its context, build or refine skills, pre-answer likely questions, reorganize memory. When a real query lands, the expensive groundwork is already done, so the response is faster and cheaper at the moment it matters.

The shared theme with skill libraries: move effort off the latency-critical path — either into a reusable artifact (a skill) or into the background (sleep-time) — so the agent improves over its lifetime without a bigger model or a weight update.

In one breath

• Most agents re-derive procedures every task; a skill library lets an agent solve once, save the solution as a reusable skill, and retrieve and compose skills later — lifelong learning in memory, not in the weights.
• Voyager is the canonical example: generate code skills, store them embedding-indexed, retrieve and compose for new tasks, with an automatic curriculum and failure-driven refinement → an ever-expanding repertoire.
• This is procedural memory (how to do things), distinct from episodic/semantic memory; it makes capability compound — repeats become cheap reuse (2.7× in the demo).
• Sleep-time compute complements it: use idle time to pre-digest context, build skills, and pre-answer likely queries, moving work off the latency-critical path.
• Both grow capability without retraining — but a skill library needs curation (dedupe, version, test, prune) or it fills with unreliable skills.

Quick check

Next

Skill libraries are procedural memory; revisit agent memory for the episodic/semantic kinds, reflection for the failure-driven refinement that improves skills, and self-improving agents for agents that bootstrap their own capability.