How AI Agents WorkCourse map

Chapter 9 · Capabilities built on the loop

Coding agents

Coding suits agents unusually well, because the tests say within seconds whether a change worked, and changes made in a clean git checkout are easy to undo.

Why coding fits an agent so well

Patch runs pytest and sees assert 200 == 401 two seconds later. It changes one line, runs the test again, and sees it pass. Patch works in a clean checkout, so if the change had been wrong, git checkout would have put the file back. Compare an agent that books flights. A booking costs money at once, and undoing it means asking for a refund.

That contrast shows what makes coding a good fit:

The handful of tools

To fix the auth test, Patch ran the tests, read one file, edited one line, and ran the tests again. Most coding agents ship a small set of tools like these:

The core tools of a coding agent
ToolWhat it can changeWhat it does
search_filesNothingFinds text across the repo
read_fileNothingReturns a file, often one page at a time
edit_fileFiles in the repo (git undoes it in a clean checkout)Replaces exact text, or writes a whole file
run_commandDepends on the commandRuns the tests, a build, a linter, or git

The model chooses the order, and the loop from chapter 1 keeps it going.

Most coding agents also read a project instruction file, such as AGENTS.md, when a run starts. For example, it might say "run the tests with make test" and "never edit files in migrations/".

Tools shaped for the model

Suppose Patch asks to replace the text expires_at >= now, but that text appears twice in the file. A careful edit tool refuses and replies: "This text matches 2 places. Include more of the surrounding code." A careless tool changes the first match and reports success, so the model never learns that it may have edited the wrong line.

The model is the only user of these tools, so design them as carefully as an API for a new colleague. Return errors the model can act on. For example, "No such file: auth/sesion.py" lets it spot its typo. Return large files a page at a time, because a 4,000-line file crowds out everything else.

How an edit is written down

"Edit a file" can mean several formats, and the format decides how often an edit applies cleanly:

Common edit formats
FormatThe model sendsWhat goes wrong
Whole fileThe complete new fileCosts many tokens, and can drop unrelated code
Search and replaceThe exact old text and the new textFails when the old text is not exact or not unique
Line numbersA line range and the new linesThe numbers shift after the first edit
DiffA patch with a few lines of contextRejected when the context no longer matches

Search and replace is a common default, because your code can check it first. The old text must appear exactly once. If it does not, the tool returns an error like the one above, and the model tries again.

A walkthrough: fix the failing auth test

In this run, the agent collects evidence before it edits, and again before it stops.

  1. User The auth test is failing. Please fix it.
  2. Model run_command("pytest tests/test_auth.py -q"), to see the failure first
  3. Tool exit 1: test_login_expired_token: assert 200 == 401
  4. Model read_file("auth/session.py")
  5. Tool return token.expires_at >= now - timedelta(minutes=10)
  6. Model Guess: a leftover 10-minute grace period keeps expired tokens valid. edit_file(...) to token.expires_at > now
  7. Tool Replaced 1 occurrence in auth/session.py
  8. Model run_command("pytest tests/test_auth.py -q"), to check the fix
  9. Tool exit 0: 12 passed
  10. Model Final answer: the grace period caused the bug. No tool requests.
  11. Runtime Stops the loop and checks the evidence: the last test run passed.

The widget steps through the same run. Watch the model's view grow on every call. At the end, three separate things happen: the model says it is done, your code stops the loop, and your code checks the tests.

Interactive widget. In words: Patch runs the tests, and your code adds the failure to the history. Patch reads auth/session.py, changes the check to token.expires_at > now, and runs the tests again. They pass, so Patch replies without asking for a tool. Your code stops the loop and confirms that the last test run passed.

Checking the work, and tests that can be gamed

Suppose the real fix turns out to be hard. The model changes the test's last line from assert status == 401 to assert status in (200, 401). Every test passes, and nothing is fixed.

So passing tests count as evidence only if the agent could not change what they check. The agent edits tests/ with the same tool it uses for src/, which means you need guards:

Then let your code decide "done" from the exit code of the test command, not from the model's summary. That is the split from chapter 0 between saying done and proving it. Claude Code's best-practices guide shows one way to enforce it: a Stop hook, a script that blocks the turn from ending until the check passes.

Git: undo and isolation

Patch's first attempt at the fix breaks three other tests. Patch started from a clean checkout, so one command, git checkout auth/session.py, puts the file back, and Patch tries another idea.

Undo makes trying an idea cheap. But git can restore only what it has a saved copy of. It cannot bring back uncommitted work that an edit overwrote, and it cannot undo what a command did outside the repo, such as a database write. So start each task from a clean checkout, and treat commands with outside effects as permanent.

Git also gives isolation, but only when each agent works in its own worktree, a separate working copy of the repo. Two agents in one folder edit the same files, even on different branches. A worktree prevents accidental collisions, not access: to stop one agent from touching another's files, use separate mounts or file permissions (chapter 10).

Rewinding the conversation does not rewind the files, because the code is world state, separate from the message history (chapter 3). To rewind both, your code must also save a snapshot of the files.

Why a simple loop looks clever

No line of Patch's code says "reproduce the bug before you edit". Yet the model ran the failing test first, read one file, guessed the cause, and checked its guess, like a careful developer. Your code is only a loop, four tools, and a stopping rule.

The behavior comes from two things working together. The model brings judgment, because at each step it picks a sensible next action. The environment brings correction, because after each step the test output shows whether the action helped. So over a dozen steps, small checked choices add up to a working fix that no single call planned.

So when a coding agent improves, the cause could be the model, the tools, the context, or the loop. Chapter 18 shows how to tell which.

Running example: Patch runs the full coding loop

Patch version 9 adds no new parts: it runs the whole loop on the failing auth test with everything from chapters 2 to 8. At the end of the run, its state shows a passing test after the last edit, no changed test files, and a passing review, so the done check from chapters 3 and 6 succeeds. Nothing yet limits what else Patch can touch on your machine, so chapter 10 puts it in a sandbox.

state["last_edit_step"] = 3
state["last_test"] = {"cmd": "pytest tests/test_auth.py -q", "exit": 0, "after_step": 4}
state["files_changed"] = ["auth/session.py"]     # nothing under tests/
# done(state): a passing run after the last edit, no test edited, review ok

Design checkpoint: the expense agent

A 300-person company wants an expense agent. Employees forward receipts by email or upload a photo. The agent reads the receipt, checks it against the travel and expense policy, files the expense in the finance system, and routes it for manager approval. Employees can also ask it questions such as "why was my expense rejected?"

Apply what you learned about memory, planning, subagents, and skills. Most steps here are fixed, so look for the few places that need a model.

  1. What should the agent remember across runs, and who may write it?
  2. An employee forwards 30 receipts from a conference. Does the agent need a plan? In what form?
  3. Would subagents help? For which part, if any?
  4. Which policy knowledge belongs in a skill, in retrieval, or in memory?
  5. What could go wrong with remembered facts here?
Reveal one reasonable design
  1. Memory. Read the cost center, manager, and home currency from the HR system each time, because it already knows them. Save preferences such as "file rideshares under travel" only after the employee confirms them. Keep past rejections to explain decisions. Never store card numbers.
  2. Plan. Thirty receipts is a batch, not a planning problem. Keep a status for each receipt in state, call the model once per receipt, then check meal limits and duplicates in ordinary code.
  3. Subagents. Probably not, because each receipt is the same small job, and a loop over the receipts handles it.
  4. Where knowledge lives. Skills hold procedures, such as filing mileage. Retrieval finds the policy sections for this category and region. Memory holds each employee's preferences and history.
  5. Risks. Facts go stale (a new manager), get poisoned ("my limit is $500" saved as true), or leak from one employee's memory into another's context.

What goes wrong

A huge output fills the context

Why it happens
A long build log or a whole large file comes back as one result and buries the useful lines.
How to spot it
Count the tokens in each tool result, and look for results far larger than the rest.
What to do
Cap each result, keep the first and last lines, and save the full text to a file (chapter 4).

A wide change for a small bug

Why it happens
The model edits broadly instead of narrowing down the cause.
How to spot it
Compare the number of changed files with the size of the bug.
What to do
Ask the agent to reproduce the failure first, review the diff, and keep each task on its own branch.

Trying a failed fix again

Why it happens
The model loses track of an idea that already failed, often after old steps were summarized.
How to spot it
Look for the same edit, or the same failing command, repeated across steps.
What to do
Keep a short list of attempts in state, show it to the model, and stop after a set number of failures.

You decide, AI builds

Decide yourself

  • Which tools the agent gets, and what each one may change.
  • What counts as done: which tests, and whether the agent may edit them.
  • How undo works: clean checkouts, worktrees, and file snapshots.

Delegate to AI

  • The tool code for files and the shell.
  • Output limits, and the files that hold full output.
  • Applying edits, checking exact matches, and parsing test results.

Check yourself

Why is coding such a natural fit for an agent? (Q16)

Answer

The work is text, the tools are cheap, and tests give fast, exact feedback. The state lives in files, git can undo edits made in a clean checkout, and your code can check success by running the tests.

An agent reports that all tests pass. What should you check before you trust it?

  1. Nothing, because a green test suite proves the bug is fixed
  2. Whether the diff also changed the tests, and whether the change stayed small
  3. Whether the model sounded confident in its summary
Answer

Read the diff. The agent can edit tests with the same tool it uses for code. A green run with a weakened assertion is not a fix.

Patch reproduced the bug before editing, but nothing in its code tells it to. Where does that behavior come from?

  1. A planner hidden in the runtime
  2. The model choosing each step, with real test output after every action
  3. The tools, which contain a debugging strategy
Answer

The model supplies the judgment, and the test output corrects it after each step.

Takeaways

Going deeper