Skill libraries & sleep-time compute

Most agents have amnesia about how to do things. Solve a task today, and tomorrow the same kind of task is approached from scratch — the same reasoning re-derived, the same tokens (the small text chunks a model reads and bills by) re-spent. Humans don’t work that way: we learn a skill once and reuse it forever, building a repertoire. A skill library gives an agent that ability — solve a task once, save the solution as a reusable skill, then retrieve and compose skills for future tasks. It is lifelong learning that happens in the agent’s memory, not in its weights (the numbers inside the model that training adjusts).

Voyager: the canonical example

The clearest demonstration is Voyager, an agent that plays Minecraft open-endedly. Its loop:

• Generate a skill. When it solves a task, it writes the solution as an executable code function (e.g. craftStoneSword()), not a one-off chat reply.
• Store it. The skill goes into a library, indexed by an embedding of its description (a list of numbers that places similar meanings close together), so it can be found later by meaning.
• Retrieve and compose. For a new task, it embeds the task, retrieves the most relevant skills, and composes them — building complex behaviors out of saved simpler ones.
• Curriculum + refinement. An automatic curriculum (the agent proposes its own next tasks, each a bit harder) keeps stretching it. When a skill fails, the agent is prompted again with the error message to refine it, a cousin of reflection.

The result is an ever-expanding repertoire: Voyager kept acquiring new abilities and generalized to new worlds far better than agents without a persistent skill store. The idea generalizes well beyond games — a coding agent that saves a helper function once and reuses it, an RPA (robotic process automation, software that clicks through business apps) agent that banks a verified procedure. This is procedural memory (how to do things), distinct from the episodic/semantic memory in agent memory.

The payoff: solve once, reuse cheaply

Re-deriving a procedure costs a full reasoning round; retrieving a saved skill is nearly free. Across repeated task types, the library turns most work into cheap reuse:

gen_cost, reuse_cost = 100, 10        # cost to generate a new skill vs reuse one
tasks = ["nav", "mine", "craft", "nav", "craft", "mine", "nav", "craft", "mine", "nav"]

without = len(tasks) * gen_cost        # no library: derive every task from scratch
seen, with_lib = set(), 0
for t in tasks:
    if t in seen:
        with_lib += reuse_cost          # retrieve & reuse the saved skill
    else:
        with_lib += gen_cost            # first of its kind: generate and save
        seen.add(t)

print(f"without library: {without}")
print(f"with library   : {with_lib}  ({len(seen)} skills generated, {len(tasks) - len(seen)} reused)")
print(f"reduction: {without / with_lib:.1f}x")

without library: 1000
with library   : 370  (3 skills generated, 7 reused)
reduction: 2.7x

Only the first task of each kind pays the full generation cost; the seven repeats reuse saved skills cheaply. The more an agent runs in a domain, the more its library covers and the cheaper it gets — capability that compounds, with no retraining.

Sleep-time compute: work off the query path

A complementary idea: not all the agent’s thinking has to happen when a query arrives. Sleep-time compute uses the agent’s idle time to do work in advance — digest and re-summarize its context, build or refine skills, pre-answer likely questions, reorganize memory. When a real query lands, the expensive groundwork is already done, so the response is faster and cheaper at the moment it matters.

The shared theme with skill libraries: move effort off the latency-critical path — either into a reusable artifact (a skill) or into the background (sleep-time) — so the agent improves over its lifetime without a bigger model or a weight update.

In one breath

• Most agents re-derive procedures every task; a skill library lets an agent solve once, save the solution as a reusable skill, and retrieve and compose skills later — lifelong learning in memory, not in the weights.
• Voyager is the canonical example: generate code skills, store them embedding-indexed, retrieve and compose for new tasks, with an automatic curriculum and failure-driven refinement → an ever-expanding repertoire.
• This is procedural memory (how to do things), distinct from episodic/semantic memory; it makes capability compound — repeats become cheap reuse (2.7× in the demo).
• Sleep-time compute complements it: use idle time to pre-digest context, build skills, and pre-answer likely queries, moving work off the latency-critical path.
• Both grow capability without retraining — but a skill library needs curation (dedupe, version, test, prune) or it fills with unreliable skills.

Quick check

Next

Skill libraries are procedural memory; revisit agent memory for the episodic/semantic kinds, reflection for the failure-driven refinement that improves skills, and self-improving agents for agents that bootstrap their own capability.