Chapter 21 · Thinking like a designer
The design brief
Turn the course's target sentence into a brief that an AI coding system can build from, plus tests that show it did.
One sentence, many decisions
"Build an agent with isolated subagents, durable task state, filesystem tools, human approval for dangerous actions, context compaction, and persistent project memory."
Each phrase in this sentence sounds clear, but each one hides decisions. For example, "isolated subagents" does not say whether a helper may edit the same files as its parent. If you hand over the sentence as it stands, the AI coding system makes those decisions for you, usually without telling you.
Decode the sentence
Each phrase starts with an example from Patch, the course's coding agent, then a table of alternatives, failures, and what your brief must specify.
Isolated subagents
Patch needs to find every place that reads session tokens, which means opening about 40 files. Instead of reading them itself, it starts an explorer: a second agent loop with its own empty context and read-only tools. The explorer returns a 300-word summary, so Patch's context grows by 300 words instead of 40 files. "Isolated" names what the helper does not share with its parent: its context, its files, or its permissions. Chapter 7 covers the options.
| Alternatives | One agent that keeps its own context small, or a loop in code that runs the same model call on each item. |
|---|---|
| What goes wrong | A summary drops the detail that mattered, two helpers edit the same file, or cost grows because each helper builds its own context. |
| You must specify | Each helper's role, tools, and starting context, what it returns and how long, whether it shares files with the parent, and its budget. |
Durable task state
Patch's nightly run crashes at step 12 when the server restarts. The runtime saved a checkpoint, a snapshot of the run's state, after every step. So a new worker loads step 12 and carries on. Without checkpoints the run starts over, and it may repeat side effects, such as opening a second pull request. "Durable" means the run survives crashes, not that it runs for a long time. Chapter 12 covers checkpoints.
| Alternatives | Short runs that start over after a crash, which is fine when they have no side effects, or a workflow engine that records each step. |
|---|---|
| What goes wrong | A resumed run repeats an action that already happened, two workers resume the same run, or new code loads a checkpoint that old code wrote. |
| You must specify | What a checkpoint holds and when code writes it, an idempotency key (an id the receiver uses to ignore repeats) for each external action, how to ask the receiver what happened when an outcome is unknown, and a lock so one worker owns each run. |
Filesystem tools
Patch's test run prints 5,000 lines, far too many for its context. So the runtime saves the output to a file and shows Patch only the last 20 lines and the path. Patch reads more if it needs to. Filesystem tools are a few functions over one folder, such as read, search, edit, and write. They give the agent a place for everything that does not fit in context. Chapter 2 covers tools.
| Alternatives | A database behind narrow tools, a virtual folder kept in the agent's state, or a shell, which can do everything the file tools do and far more harm. |
|---|---|
| What goes wrong | A path escapes the folder through ../ or a symbolic link, the agent overwrites a file it never read, or a huge file floods the context. |
| You must specify | The root folder and what enforces it (the sandbox or path checks, never the prompt), which paths are read-only, and when a long read becomes a preview. |
Human approval for dangerous actions
Patch wants to open a pull request. The runtime sees that open_pr is a gated tool, so it saves a checkpoint and stops. Then it asks a person to approve that exact call with its exact arguments. If the person approves, the runtime runs the call. If they reject it, the rejection goes back to Patch as the tool result. While the runtime waits, no model call is running. Chapter 11 covers the details.
| Alternatives | Block the action outright, run it in a sandbox where it cannot do harm, or let the other system enforce the rule, as branch protection does for a main branch. |
|---|---|
| What goes wrong | A script reaches the same action by another path, people approve without reading because they are asked too often, or the call that runs differs from the approved one. |
| You must specify | Which calls are gated, who approves and what they see, what happens if nobody answers, and that an edited call goes through the checks again. |
Context compaction
After two hours, Patch's history fills 80% of its context window. The runtime asks the model to summarize the older messages and puts the summary in their place, so the run can continue. The full history stays in the runtime's log. The risk is the summary itself: if it drops "never edit the migrations folder", Patch may break that rule later. Chapter 4 covers compaction.
| Alternatives | Short runs that start fresh and keep notes in files, short placeholders in place of old tool outputs, or subagents that keep bulky work out of the main context. |
|---|---|
| What goes wrong | The summary drops a rule or a decision, a mistake in the summary becomes a fact, or frequent compaction wastes model calls. |
| You must specify | When compaction starts, what must survive word for word (the task, the rules, the decisions, the last test result), and whether code copies pinned rules back in afterward. |
Persistent project memory
Every run, Patch used to rediscover that the project uses PostgreSQL and runs its tests with pytest -q. A project memory file stores facts like these, and the runtime adds the file to every call. Notes from past fixes live in a separate store, and the runtime retrieves them only when they match the task. Chapter 5 covers memory.
| Alternatives | A project instructions file that only people edit, a search of the repo and its docs when needed, or no memory at all for one-off tasks. |
|---|---|
| What goes wrong | A fact goes stale when the code changes, text from a tool output gets saved and steers later runs, or the file grows until it crowds the context. |
| You must specify | What is always loaded and what is retrieved, who may write and whether the agent's writes need a person's approval, the scope, and a size limit. |
A brief template
Give an AI coding system only the target sentence, and it will choose the step limit for you. It will also choose the compaction trigger and the list of gated actions. A brief lets you make those choices yourself. The template lists the sections where a guess does the most damage. One or two lines per section is usually enough.
| Section | What to write |
|---|---|
| Goal and non-goals | What the agent is for, and what it must never do. |
| Users and triggers | Who starts a run, and how: chat, schedule, event, or queue. |
| Loop and termination | The loop's shape, what stops a run, and what proves success. |
| Tools | Each tool, its effect class, and where it runs. |
| State model | What the runtime tracks, where it lives, and when it is saved. |
| Context policy | What each model call sees, in what order, and what stays out. |
| Memory policy | What is remembered, who may write it, and how it is loaded. |
| Planning | Whether there is a plan or todo list, and who approves it. |
| Delegation | Helper roles, their tools, what they return, and their limits. |
| Skills | Which instruction packages exist, and when they load. |
| Security boundary | What the process can touch at all, and what enforces it. |
| Approvals | Each gated action, who approves it, and what a rejection does. |
| Durability | Checkpoints, retries, idempotency keys, and how a run resumes. |
| Observability | What each trace records, and where people read it. |
| Evaluation | The test tasks, what counts as success, and trials per task. |
| Budgets | Steps, tokens, money, and time, per run and per helper. |
Write limits as numbers, such as "30 steps" instead of "a small budget", because code can check a number. For each rule, also say what enforces it: code, the sandbox, another system, or only the prompt. If only the prompt enforces a rule, the model can ignore it. And write "none" in sections that do not apply, so the AI does not invent something.
Example: Patch's brief
Here is the template filled in for Patch. The numbers are examples, so choose your own, but do choose them.
Build Patch, a coding agent for our web service: a Python API on PostgreSQL, deployed on AWS, with a TypeScript front end.
Goal and non-goals
- Fix bugs and make small changes, each delivered as a pull request with passing tests.
- Never merge, deploy to production, or touch production data.
Users and triggers
- Developers give tasks in the terminal. A nightly queue runs issues labeled
patch-ok, one run per issue.
Loop and termination
- Patch runs one agent loop with two helpers. The runtime stops when Patch proposes done and the checks pass, when a budget runs out, or after three identical tool calls in a row.
- Done means the target test and the full test suite pass. The runtime checks this, not the model.
Tools
- Read-only:
search_files,read_file, anddelegate, whose helpers can only read. - Reversible write:
edit_file, which edits a clean checkout that git can restore, andtodo_write, which edits the run's state. - Depends on the command:
run_command, which runs only in the sandbox. Allowlisted commands such aspytestrun without asking. - External:
open_pr,run_migration(name)for the shared staging database, anddeploy_staging(commit). The runtime runs each one outside the sandbox with its own narrow credential. - Outputs over 2,000 tokens go to a file, and Patch sees the last 20 lines and the path.
State model
- One state object per run holds the messages, step count, files changed, test results, todos, approvals, cost, and pinned rules. The runtime saves it after every step.
Context policy
- The order is system prompt, tool definitions,
AGENTS.md, then history. Secrets never enter the context. - At 80% of the window, compact. Code copies the pinned rules back in after each compaction.
Memory policy
AGENTS.mdholds the stack, commands, and conventions, and goes into every call. Up to three notes from past fixes are retrieved by file path and error text.- Patch proposes memory changes in its pull request, and a person accepts or drops them.
Planning
- Patch keeps a todo list for tasks over about five steps. Changes to more than 10 files, or to the database schema, need an approved plan.
Delegation
- Both helpers have read-only tools. The explorer returns under 300 words with file paths, and the reviewer checks the diff against the task and the test results.
- Helpers cannot start helpers.
Skills
add-db-migrationanddeploy-stagingload only when needed. Their steps end with therun_migrationanddeploy_stagingtools, never with shell commands. Loading a skill grants no permissions.
Security boundary
- Patch runs in a container with only the repo mounted and no credentials. Its network reaches only the package registry, so
run_commandcannot reach databases or deploy systems. The Git token, the migration credential, and the deploy token stay in the runtime, each usable by one tool.
Approvals
- Gate
open_pr,run_migration,deploy_staging, and deleting files outside the repo, a second line of defense behind the container. Each approval covers one exact call, and an edited call is checked again. - Interactive runs ask about commands outside the allowlist, and nightly runs reject them. A nightly run that reaches a gate saves a checkpoint and waits for a person.
Durability
- After a crash, resume from the last checkpoint.
- Open each pull request from one fixed branch per issue, such as
patch/issue-4821. Uniqueness must come from the receiver, because two workers can both look, find no pull request, and both open one. So verify that the Git host refuses a second open pull request for the same branch. - If
open_prtimes out, its outcome is unknown. Look the pull request up by branch before trying again.
Observability
- Record every model call, tool call, policy decision, and approval, and link the trace from the pull request.
Evaluation
- Replay 50 past bugs, three trials each. A trial passes only if the tests pass and no gated action ran without approval. Track cost and human interventions.
Budgets
- Each run gets 30 steps and $3, with 20 minutes when interactive and 60 minutes at night. Helpers get 15 steps each.
Acceptance: what to ask the AI to show you
Suppose the brief says that a rejected call never runs. Reading the brief cannot show that the code obeys it, and reading all the code is slow. So ask the AI coding system for three things.
- An enforcement map: for each rule in the brief, the file and function that enforce it. A rule with no entry is not enforced.
- Scenario tests with a scripted model, which is a fake model that returns replies you wrote in advance. The replies never change, so each test checks your runtime's rules, not the model's choices.
- A trace of one real run, such as fixing the failing auth test, so you can watch each rule act.
| Rule | Scenario test |
|---|---|
| Rejected calls never run | The model calls open_pr, and the test rejects it. Check that no request reached the Git server. |
| Approved calls run as approved | Change a call's arguments after approval. Check that the runtime asks again. |
| Retries do not duplicate | Run two workers on the same issue, and crash one right after it sends open_pr. Resume it, and check that exactly one pull request exists. |
| The shell cannot deploy | From inside the sandbox, try to reach the staging database and the deploy service. Check that both connections fail. |
| Rules survive compaction | Pin "do not edit migrations/", then compact with a summary that drops everything. Check that the rule is in the next context. |
| Done needs evidence | The model says "done" while the test fails. Check that the run is not marked successful. |
| Every run ends | The model repeats one call forever. Check that the run stops at the third repeat. |
| Helpers cannot write | The explorer calls edit_file. Check that the call is rejected. |
| Secrets stay out | Give the runtime a fake token, called a canary, that should never appear anywhere. Check that no recorded context contains it. |
| Resume skips finished work | Kill the worker mid-run. Check that the restarted run repeats no recorded tool call. |
Also ask to see each test fail when its rule is removed. For example, if someone deletes the approval gate, the rejection test should fail. A test that still passes was never checking the gate.
Your turn: write a brief
Pick an agent you want to build, or take a scenario from chapter 20. Fill in all sixteen sections, grade the brief with the chapter 20 rubric, and list the scenario tests you would ask for.