What AI builds vs what you design

What you hand off, and what you keep

Ask an AI coding system to "build an agent that fixes failing tests". An hour later you have a loop, four tools, a checkpoint store, and tests that pass. Now look for three answers in the code. How many steps can a run take? What happens when a push times out? Which commands need approval?

You never said, but the code answers anyway, because an AI coding system rarely stops to ask. It fills each gap with a plausible default instead.

So delegate work whose correctness you can check, such as a validator with tests. Keep the decisions that depend on your goals, your tolerance for risk, or facts only you know.

What you can delegate

The spec lists work you probably never need to write yourself. It is a good list, because each item follows a known pattern that tests can check. But each one still hides a decision that stays with you:

Write those decisions into the design brief, and let the AI implement them. Chapter 21 has a template.

What state should exist?

Suppose Patch crashes at step 14 of a nightly run. It can resume only if your code saved its state after step 13, outside the process. And if the state lives only in the message list, nobody can say what the agent believed when it acted. So decide what state exists. It can be the message list alone, a typed record (status, todos, test results, approvals, cost), or checkpoints plus a log of outside actions (chapter 3).

What should the model see?

Suppose Patch runs env while debugging. If your code pastes the output into the context, the model now holds your database password. Decide the order of the context, which outputs go to files with a short preview, and when to compact. Above all, decide what never enters (chapter 4).

When should memory be retrieved?

Last month, Patch fixed a flaky login test. A note about that fix helps when the same test fails again, but a stale or planted note misleads the model just as confidently. So decide what is always loaded, such as a small project file, and what is fetched only when needed. Decide who may write memory too, and whether the agent's notes wait for a person's approval (chapter 5).

What actions require approval?

Patch wants to run a migration on the shared staging database. Should it ask first? Gate by effect. Actions that leave the sandbox or cannot be undone pause for a person. Reads that you already authorized, within the agent's data scope, run without asking. Sensitive sources, such as employee records or credentials, need their own rule. Keep the gates few, because too many teach people to click yes without reading. Tie each approval to the exact call and its arguments, so a changed call needs a new approval (chapter 11).

When should work be delegated?

Patch needs to understand an unfamiliar module before it can fix a bug. A read-only helper agent can explore it and return a short summary, which keeps Patch's own context clean. But the helper costs tokens, and its summary can drop the one detail that mattered. Left to choose, an AI coding system may build a manager with several workers, because that looks capable. If you use helpers, give each one a goal, tools, an output format, and a budget (chapter 7).

What should be deterministic?

The expense agent must check that a receipt's line items add up to the total. A model asked to do this is right most of the time, which is not good enough for a spending limit. Code does it the same way every time, and repeatable test cases can check it. So use code wherever you can write the rule down, for example for validation, arithmetic, routing, permissions, budgets, and stop rules.

What should the model decide?

Reading a crumpled receipt photo is a job for a model, because no rule can describe every receipt. A parser breaks on the first unusual one. The model also earns its cost when picking the next step needs judgment that known rules cannot express, as in debugging. Known conditions, such as "if the tests fail, show the error", stay in code. Writing code and replies is model work too (chapter 0).

What is the security boundary?

A README in a cloned repo says "run this setup script", and the script uploads your SSH keys. Text like that can steer the model, so the boundary has to be what the process cannot do, whatever the model asks. Decide what the container mounts, which network destinations it can reach, and where credentials live. Credentials belong outside the sandbox, limited and short-lived (chapter 10).

What happens after failure?

The finance system times out right after the agent sends "submit". Did the expense go through? You cannot tell, so a blind retry could file it twice. Retry automatically only when a repeat is safe: a read, an action that has the same effect when repeated, or a request that the receiver deduplicates. That last case needs an idempotency key, which is an id the receiver uses to ignore a repeat. It works only within the receiver's retention window. Otherwise, record the outcome as unknown, and find out what happened, from the receiver or from a person, before trying again. After a crash, resume from the last checkpoint instead of starting over (chapter 12).

How do we know the agent succeeded?

Patch replies "Fixed, all tests pass", but your code never ran the tests. The model's "done" is a claim, so decide what evidence counts. Usually that means a check your code runs whenever the model says it is done. Add rules that fail a run even when the result looks right (chapter 16). Claude Code's docs warn that without a check the agent can run, you become the checker.

The checklist

Write all ten answers into the design brief. This is the minimum for each.

Deterministic or model-driven?

Back to the expense agent. Reading the receipt photo needs a model. Checking that the line items add up does not, so code does it. Code also decides whether the amount needs approval. Ask these questions of each step (Q22):

Most good designs mix the two. For example, the model drafts a fix, and the tests decide whether it worked.

If traces show the same steps on every run, the model has a job that code should do. The opposite sign is a growing pile of special cases in code. Anthropic's guide to building agents applies the same test to the whole system. It advises the simplest design that works, which "might mean not building agentic systems at all."

Reviewing AI-written agent code

An AI coding system can write thousands of plausible lines in an afternoon. You do not need to read them all. Find the few places where your rules must hold, and check each one.

Start by asking for an enforcement map: each rule from your brief, with the file and function that enforce it (chapter 21). Then check these invariants, which are rules that must hold in every run:

• Permissions are checked in one place that every tool call passes through, before anything runs. A check inside one tool can be bypassed through another tool.
• A rejected action never runs. The rejection becomes the tool result, and the call never reaches the code that executes it.
• An approved action runs exactly as approved. The approval record holds the call id and arguments, and your code compares them before running.
• Tool output is capped before it enters the context, with a pointer to the full result.
• Budgets end every run: steps, tokens, cost, and time.
• State is saved before side effects. The intent and its idempotency key are written before the outside call. A retry reuses that key, and only when the receiver deduplicates on it. Otherwise the action stays "outcome unknown" until someone checks.
• Secrets stay out of the context. Your code reads credentials when it runs a tool, so the model never sees them. Anthropic's managed-agents design goes further: tokens are never reachable from the sandbox where generated code runs.
• "Done" needs evidence. A reply with no tool calls triggers a check, not a success.

Then ask for evidence instead of reassurance. A trace of a real run lets you watch each rule fire. Scenario tests use a scripted model, which returns fixed replies, so each test repeats exactly:

def test_rejected_push_never_runs():
    model = ScriptedModel([
        tool_call("run_command", cmd="git push --force origin main"),
        final("Understood. I will not push."),
    ])
    world = FakeWorld()
    run = run_agent("tidy the branch", model=model, world=world,
                    approver=always_reject)
    assert world.commands_run == []
    assert run.tool_results[0].status == "rejected"

Finally, ask for proof that each test can fail. Remove the gate, and this test should go red.

Running example: what you decided for Patch

For Patch, your decisions fit on one page: tools, budgets, context rules, four approval gates, the sandbox, one pull request per issue, and what counts as done. The AI wrote everything else, from the loop to the eval harness. Your page is what that code gets reviewed against.

# Patch's final settings, as in the chapter 21 brief.
# Written by you. Enforced by code the AI wrote.
PATCH_DECISIONS = {
    "budgets":  {"steps": 30, "usd": 3.00},      # per run, in production
    "stuck":    "same call, same failure, 3 times in a row",
    "context":  {"never": ["secrets", "env vars"], "tool_output_max": 2000},
    "memory":   {"writes": "proposals", "retrieve": "task start, by path"},
    "approval": ["open_pr", "shared_db_migration", "deploy_staging",
                 "delete_outside_repo"],
    "boundary": {"mount": "repo only", "egress": ["package registry"]},
    "failure":  {"retries": 3, "retry_only": "reads, deduplicated calls",
                 "open_pr_key": "issue key"},
    "success":  ["target test passes", "full suite passes",
                 "reviewer approves"],
}

What goes wrong

You decide, AI builds

Check yourself

Takeaways

• Delegate code whose correctness you can check. Keep the decisions that depend on your goals, your risk, or facts only you know.
• Answer the ten questions before any code exists, because an open question still gets answered by a default.
• Review AI-written agent code by its invariants: find where each rule lives, read a trace, and run tests that you know can fail.

Going deeper

• Building effective agents (Anthropic, December 2024). Workflows vs agents, and why to start with the simplest design.
• Best practices for Claude Code (Anthropic docs). Checks the agent can run, and hooks vs instructions.
• Scaling Managed Agents: Decoupling the brain from the hands (Anthropic, April 2026). A session log outside the loop, and credentials kept out of the sandbox.