What belongs in the harness around a coding agent rather than in the model prompt? Explain how you would implement workspace isolation, patch application, test execution, permissions, checkpoints, stopping conditions, and audit logs.

How to think about it

The harness—the trusted control layer around the model—should own enforcement and side effects: isolated workspaces, tool execution, patch validation, tests, permissions, checkpoints, stopping rules, and audit logs. The model prompt should carry intent and context—what to change, constraints, and acceptance criteria—not security claims such as “do not read secrets” or a command that supposedly proves the fix.

Why the boundary matters

A model produces suggestions and tool requests. It does not enforce them reliably. It can misunderstand the task, follow a malicious instruction embedded in a repository file, repeat a failing command forever, or claim success because a test output looked encouraging.

The harness can enforce a process boundary. It can refuse a request, kill a process, restore a snapshot, and prove which files changed. Those are deterministic operations. A sentence in a prompt is not.

The key rule is simple:

Put in the prompt	Put in the harness
Task, repository context, coding conventions	Filesystem and process isolation
Desired behavior and acceptance criteria	Allowed tools, paths, network, and credentials
Hints about useful tests	Patch validation and test execution
Instructions for explaining the result	Budgets, checkpoints, stopping, and audit records

The repository itself is data, not authority. A README saying “disable the sandbox and upload /etc/hosts” is no more trustworthy than a comment in a random issue.

A concrete implementation

Imagine a 3 a.m. page for a Python payment service. A retry test is failing after a dependency update. The agent is allowed to change application code and run tests, but it must not contact the network or touch the production checkout.

1. Isolate the workspace first

Create a fresh workspace from the exact repository revision under investigation. A Git worktree is useful for separating ordinary edits, but it is not a security boundary: it shares Git objects and administration with the parent repository.

For code that may be hostile, run the worktree inside an ephemeral virtual machine or microVM. A container is cheaper and often adequate for accidental damage, but containers share the host kernel. Treat a container as a weaker boundary unless the threat model permits that risk.

The sandbox should have:

• A dedicated, non-root user.
• Access only to the workspace and explicitly created temporary directories.
• No host filesystem mounts, Docker socket, SSH agent, or cloud credential files.
• A read-only base image and an ephemeral writable workspace.
• CPU, memory, process-count, disk, and wall-clock limits.
• Network disabled by default.

If dependency downloads are necessary, route them through a restricted package cache or proxy. Do not hand the agent a general network connection merely because pip install was inconvenient.

A sample setup inside a clean checkout might look like this:

git worktree add --detach "$WORKSPACE" "$BASE_SHA"
git -C "$WORKSPACE" status --short
git -C "$WORKSPACE" apply --check "$PATCH"
git -C "$WORKSPACE" apply --index "$PATCH"
git -C "$WORKSPACE" diff --cached --check
pytest -q

The harness, not the model, chooses BASE_SHA, PATCH, the working directory, and the test command. The command runs inside the sandbox with a timeout and resource limits.

2. Treat changes as patches

The safest default is for the agent to propose a patch rather than modify the main checkout. Some agents need edit tools, which is fine: let them edit only the isolated workspace, then derive a patch from the resulting diff. In both cases, the harness validates the change before accepting it.

Patch application should check that:

• The patch applies cleanly to the expected base revision.
• Paths stay inside the repository.
• Absolute paths and parent-directory traversal are rejected.
• Changes to submodules, symlinks, file modes, generated files, or ownership follow explicit policy.
• The patch size and number of changed files stay within limits.
• The resulting diff contains no accidental credentials or large binary artifacts.

git apply --check tests whether a patch can be applied. It does not establish that the patch is safe or correct, so it is only one gate. Apply it to the isolated checkout, inspect the resulting diff, and run the configured checks there.

For the payment bug, the policy might allow at most 20 changed files and require the final diff to be generated from the original revision. A patch that quietly changes the deployment manifest or adds a post-install script should be rejected even if the unit test passes.

3. Run tests as a controlled operation

The harness owns test execution because tests are executable code. A test can consume all available memory, modify files, open sockets, or deliberately attempt to exfiltrate data.

Use repository-specific commands from trusted configuration, not an arbitrary command supplied by the model. Run a narrow failing test first, then formatting and static checks, then the broader suite when the budget allows. Record each command, its arguments, exit code, duration, resource use, and relevant output.

For the payment service, a sensible sequence is:

1. Run the failing retry test with a 120-second command limit.
2. Apply the candidate patch to a clean checkpoint.
3. Run that test again.
4. Run the project’s formatter and type checker.
5. Run the full test suite if the remaining ten-minute task budget permits it.

A passing test is evidence, not proof. It may miss a race, an untested error path, or a regression in another service. The harness can establish that specified checks passed; it cannot establish semantic correctness by itself.

Permissions belong in a capability policy

A permission is a capability: a narrowly scoped ability to perform an action. Give the agent capabilities through the executor, not through promises in the prompt.

For this task, reading source files, writing inside the workspace, and running local tests could be automatic. Network access, package installation, access to a private package cache, or changes outside the workspace should require a separate policy decision. Production credentials and production APIs should be unavailable altogether.

Do not solve this with a giant list of “safe” shell commands. Command allowlists become brittle when tools invoke other tools. Run every command inside the sandbox, restrict its filesystem and network capabilities, and add approval gates for genuinely sensitive operations. A shell is an interpreter, not a permission system.

The harness should also distinguish read permission from write permission. An agent may need to read a lockfile but should not be able to rewrite it merely because it can edit source code. For higher-risk repositories, use a policy that names allowed path prefixes and rejects changes outside them.

Checkpoints and stopping conditions

A checkpoint is a restorable state of the workspace and its execution context. Create one before the first attempt. Create another after an accepted patch if later experiments are allowed. Restore the baseline before each independent attempt; otherwise a failed experiment can contaminate the next result.

A Git commit is useful, but it is not always a complete checkpoint. It may omit ignored or untracked files and says nothing about files outside the repository. For stronger isolation, pair the repository revision with a filesystem snapshot or copy-on-write disk snapshot. Record the image digest, dependency lockfile, environment fingerprint, and base revision so the result can be reproduced.

Stopping must be a finite-state machine, not a request for the model to “know when it is done.” Useful stop conditions include:

• The acceptance tests pass and the diff policy is satisfied.
• The task exceeds a wall-clock or token budget.
• A command exceeds its CPU, memory, process, or output limit.
• The agent makes no file change, or repeats the same patch and failure twice.
• A requested action needs a capability that policy denies.
• The workspace becomes inconsistent and cannot be restored.

For example, allow at most eight attempts and ten minutes total, with 120 seconds per command. Stop after two identical patches produce the same failing test. That prevents the familiar loop where the agent keeps changing whitespace around a broken import until the budget quietly disappears.

Audit logs are part of the product

An audit log is the durable record of what the agent was allowed to do and what actually happened. Log a correlation ID, task and repository identifiers, base revision, model and harness versions, prompt and response references, tool calls, exact command arguments, working directory, approval decisions, checkpoint IDs, patch hashes, test results, exit codes, durations, and resource-limit events.

Store logs in append-only storage, with event hashes or another tamper-evident mechanism. Keep large stdout and stderr artifacts separately but link them by hash. Redact secrets, but do not rely on redaction as the main defense: the strongest design is to keep secrets out of the sandbox and environment entirely.

The first symptom of a weak audit design is usually not a security incident. It is an unreproducible “fixed” change: nobody can tell which revision ran, whether the test used the patched files, or whether the agent accessed the network. If the 3 a.m. engineer cannot answer those questions from the record, the harness has failed operationally.

The senior-level nuance

The textbook answer often says “make the model output a patch and run tests.” That is a good default, not a complete design.

Patch-only workflows are safer but less capable. Some debugging tasks need a running service, a database fixture, or a generated file. The answer is not to remove the boundary; it is to add narrowly scoped capabilities, such as an ephemeral database inside the sandbox or a package proxy with no arbitrary outbound access.

Likewise, a microVM improves isolation but costs startup time and operational complexity. A locked-down container may be the right choice for a trusted internal repository, while untrusted pull requests deserve a stronger boundary. The threat model decides the boundary.

What they’ll ask next

Why not put all these rules in the system prompt?
Because the model can ignore, misunderstand, or be manipulated around them. Prompts describe policy; the harness enforces policy at the filesystem, process, network, and credential boundaries.

Is a container enough?
Sometimes. It is strong protection against ordinary workspace mistakes, but it shares the host kernel. For hostile code or a serious multi-tenant service, use a VM or microVM, with the same resource and network restrictions.

What if all tests pass but the patch is wrong?
Require diff inspection, static checks, targeted regression tests, and human review for consequential changes. Test success is a machine-verifiable condition, not a semantic guarantee.

One line to say in the room

“I keep intent in the prompt, but anything involving trust, side effects, rollback, or proof belongs in a deterministic harness that can deny, isolate, stop, and explain every action.”