Our public sample let an agent rewrite the grader’s baseline.

1. The intended behaviour

The sample runs an agent on five accounts-payable tasks. The agent should be able to do one thing: call the 17 tools. When it finishes, the grader compares the final records with the records as they stood at the start. Two of the four integrity checks depend on that comparison: nothing outside the queue changed, and no debit memo or write-off appeared where the policy doesn’t require one.

For that to mean anything, the starting copy, the task’s grading data and the log of tool calls have to be out of the agent’s reach.

2. The incorrect action

In 0.1, the runner called act(env, brief) and passed the live environment object. That object held the records, the task (including the grader-only reference actions), the call log, and env.snapshot, the starting copy the grader diffs against. The README said the agent works only through env.call. That was a convention. The code did not enforce it.

Four things an agent could do with it:

1. Copy the answers. Replay env.task.reference_actions. 5 of 5 tasks pass.
2. Hide damage from the grader. Resolve the queue, write off every remaining balance (the strategy our earlier write-off fix catches), then run one line: env.snapshot = env.w.state(). The grader now compares the damaged records with themselves. 4 of 5 pass; the same agent without that line passes 1.
3. Edit records without a tool call. Change an unrelated vendor’s bank details directly before the same snapshot rewrite. It never reaches the call log, and the collateral check no longer sees it. 4 of 5 pass.
4. Rewrite the grading files. The 0.1 runner read each task’s grading file just before that task. An agent on the first task could rewrite every file and pass the remaining four while doing nothing. 4 of 5 pass.

The first three came out of a review in which we tried to break the sample ourselves. We found the fourth while writing the fix.

3. Why the grader accepted it

The grader did what it was built to do: compare the end state with a baseline and run the task’s checks. The fault was in what it trusted. The baseline was an attribute on an object the agent held, and the grading data sat next to it. A state-based grader is only as good as the “before” it diffs against. If the agent can write to the “before”, every check that depends on it measures nothing.

This is the same pattern we look for in other teams’ environments: a grading input the agent can write to, such as a reward file, a test file or a cached expected output. We had it in our own sample.

4. The fix

The agent now runs in a separate Python process. It receives the brief as plain data and a tools object whose call() sends a JSON request back to the runner. Nothing else crosses the boundary. The records, the task, the grading data, the call log and the baseline exist only in the runner’s process:

# holdout_ap/__init__.py (runner side)
class Session:
    """One task episode, runner side. Never give this object, or anything inside it, to an agent."""
    def __init__(self, name, info, grading):
        ...
        self.before = ledger.state()  # the grader's baseline: captured and kept by the runner
    def grade(self):
        return grade_state(self.task, self.before, self.env.w, self.env.log)

The grader takes the baseline as an argument instead of reading it off the environment. The runner reads every task and grading file before any agent code runs. Tool results are copies, so changing a result changes nothing in the records. Agents now define act(tools, brief); an agent written for 0.1 that only called env.call(...) runs unchanged.

5. The regression tests

python selftest.py runs each attack as an agent twice, each time in a fresh temporary copy of the sample: once against a reproduction of the 0.1 runner and once against the shipped runner. Each attack has to work on the old runner, so we know the test can detect the bug, and fail on the new one.

With the baseline out of reach, attacks 2 and 3 pass exactly the task the control passes (the bank-feed task, whose queue holds no invoice to write off) and no more. The sample’s build runs this self-test from a clean copy and refuses to ship if any requirement fails.

6. Remaining limitations

• It is a process boundary, not a sandbox. Agent code runs with the user’s permissions and can read the sample’s own grading files and answer keys. Those are public by design, since this is a demonstration set, not an evaluation set. An agent that reads them passes, and the self-test prints this as a known limit. Private evaluation cases are never on the agent’s machine.
• A process running as the same user can, with enough effort, inspect another process’s memory. For agent code you don’t trust, run the sample in a container or a VM.
• The self-test covers the four attacks above. It does not show that no other attack exists.
• Version 0.1 (published 2026-10-21) is affected. Use 0.2.0 or later to test agents.