Architecture exercises

A deterministic workflow is enough, because the sources, the schedule, and the report format are known in advance. Code runs the steps in order, and the model only reads changed pages and writes the report.

Monday 06:00 (code)
  |
  v
For each source, in parallel (code):
  fetch --> compare with last week's copy
     |                |
     | failed         | changed
     v                v
  mark "not checked"  extract facts and quotes (model)
     |                |
     +-------+--------+
             v
Choose what matters, write the report (model)
  |
  v
Check every quote against its page (code)
  |
  v
Post once for this week (code)

Tools

• fetch_page(url) Read-only Code limits how often it calls each site.
• search_news(company, since) Read-only It finds recent news.
• save_copy(source, text) Reversible write It keeps each page for next week's comparison.
• post_report(week, text) External It posts once per week.

Another design

• An agent with web search can find new sources, but its coverage varies, so it fits a monthly run that suggests sources.

What could go wrong

• A redesign makes a whole page look changed, so the model may invent a price change. Code sends very large changes to a person.

An agent runs once per CI failure. Code decides whether to run, the model works out a fix, and code decides whether the fix may be pushed.

CI failed (webhook)
  |
  v
Opted in? Under 2 tries? (code)
  |
  v
Rerun the job once (code) --> passes: label "flaky", stop
  |
  v
Agent loop in a fresh sandbox (model): read log, edit, recheck
  |
  v
Diff check (code): passes, small, no tests weakened
  |-- ok -----> push one commit (code)
  |-- not ok -> comment with the diagnosis (code)

Tools

• get_ci_log(job) Read-only It returns the failing step's log.
• edit_file Reversible write Edits stay in the sandbox until code pushes them.
• run_check(name) Reversible write It reruns only the command CI ran, inside the sandbox. The agent has no general shell.

Another design

• Run deterministic fixers, such as the linter's auto-fix, before the agent, because they need no model.

What could go wrong

• The author pushes new commits during the run, so the fix is based on old code. Code checks the branch before pushing and drops the fix rather than overwrite the author's work.

Code routes each message, and a small agent handles anything beyond a status check. The refund limit lives in the refund tool, because only code can enforce a policy.

Message arrives (event)
  |
  v
Verify the customer, load their orders (code)
  |
  v
What is it about? (model, small and cheap)
  |-- order status --> answer from a template (code)
  |-- anything else --> support agent (model + narrow tools)
                           |
                           v  big refund, legal, 3rd contact
                        hand off to a person

Tools

• get_order(id) Read-only It returns only this customer's orders, because code takes the customer id from the login.
• update_address(order, address) Reversible write It works only before the order ships.
• issue_refund(order, amount) External It applies the refund policy itself and pays only to the original card.
• send_reply(text) External Every message to the customer goes through it.

Another design

• Forms and templates cover the common requests more cheaply. Many teams start there and add an agent later.

What could go wrong

• A refund call times out, and the retry pays twice. So code sends an idempotency key, an id the payment service uses to ignore repeats.

This needs an agent, because each search depends on what the last one found. A lead agent splits the question, and researcher subagents explore the parts in parallel. It stops when every part has notes with citations, or when the budget runs out.

Your question
  |
  v
Lead agent splits it into 3-5 parts (model)
  |
  +--> researcher 1 --+
  +--> researcher 2 --+--> notes with quotes and links (files)
  +--> researcher 3 --+
  |
  v
Lead agent merges notes, flags disagreements (model)
  |
  v
Check each quote against its source (code)
  |
  v
Report, plus suggested memory updates for you

Tools

• web_search, fetch(url) Read-only Only researchers have them. Fetch saves each page to a file.
• write_note(path, text) Reversible write It writes in this question's folder.
• delegate(question, budget) Read-only Only the lead has it. It starts a researcher with read-only tools.
• propose_memory(fact) Reversible write It adds a suggestion to your review list.

Another design

• A single agent is cheaper and fine for narrow questions. Add researchers when page text starts to fill its context.

What could go wrong

• A summary drops a caveat. The notes keep the full quotes, so the lead can check.
• Trusted sources crowd out everything else. So keep some outside results, and show what was left out.

This is mostly a workflow, because every lead goes through the same steps. The model drafts emails and explains unusual scores. Outbound email gets a gate, because a bad email cannot be unsent.

Demo request (event)
  |
  v
Remove duplicates, check consent, look up company (code)
  |
  v
Score fit: rules (code), unusual cases explained (model)
  |-- poor fit --> polite template (code)
  v
Draft an email from looked-up facts only (model)
  |
  v
Rep approves --> send and record in the CRM (code)
  |
  v
Replies --> reply agent: answers, books a call (model)

Tools

• lookup_company(domain) Read-only It gets company facts from a data provider, each with its source.
• save_draft(lead, text) Reversible write It saves a draft in the CRM, the sales team's customer database.
• send_email(draft) External It runs after approval, and code blocks anyone who did not ask to be contacted.
• book_meeting(lead, time) External It runs once the prospect picks a time.

Another design

• As a safer first step, the agent writes a brief and a draft, and the rep sends the email.

What could go wrong

• The reply rate rises because people reply angrily. So count meetings held and complaints, not replies.

Split the job. The browser agent only finds and downloads invoices. Code compares them with purchase orders, because that is arithmetic, not judgment. The model drafts disputes, and a person submits them.

Weekly schedule (code)
  |
  v
Browser agent in a fresh, isolated browser (model)
  log in (code fills in the password)
  open invoices --> filter last week --> download each one
  |
  v
Downloads match the portal's invoice count? (code)
  |
  v
Compare with purchase orders (code)
  |-- all match --> mark ready to pay
  v   mismatch
Draft a dispute (model) --> a person submits it

Tools

• read_page() Read-only It returns the accessibility tree: the page's buttons, links, and text, as a screen reader sees them. This is smaller and more exact than a screenshot.
• click(element) Depends on target Code blocks buttons such as Submit, Pay, and Delete.
• download(link) Reversible write It saves the file to the run's folder.
• login() Read-only It takes no arguments, so the model never handles the password.

Another design

• Ask the supplier for an export or an API first. If the portal rarely changes, a plain script with no model also works.

What could go wrong

• The layout changes and the agent cannot find the invoices. It stops after 15 steps and alerts a person instead of guessing.

This needs an agent, because each query depends on the last result. But every tool is read-only, each metric uses its approved definition, and every number in the answer comes from a query that ran.

Question in chat
  |
  v
Agent loop (model + read-only tools, at most 20 steps)
  look up metric --> write SQL --> run on a copy --> read result
  --> next query, until the answer holds up
  |
  v
Build the answer (code): copy numbers from results, attach SQL
  |
  v
Reply, labeled "approved metric" or "exploratory"

Tools

• get_metric(name) Read-only It returns the approved definition, such as what counts as a signup.
• search_catalog(term) Read-only It finds tables, columns, and known data problems.
• run_sql(query) Read-only It runs on a read-only copy of the database, with limits on rows, time, and cost.

Another design

• One model call that writes SQL handles lookups such as "signups last week". But it cannot follow up, so it fails on "why" questions.

What could go wrong

• A join counts rows twice and gives a believable wrong number. The recheck catches most of these.
• The answer turns a correlation into a cause. So answers separate what the data shows from guesses about why.

The agent only reads by default, and it keeps a list of hypotheses with evidence for and against. A cause counts as found only when it explains the symptom and when it started, and a fix confirms it.

Alert fires (event)
  |
  v
Gather recent deploys and past incidents (code)
  |
  v
Investigate (model + read-only tools, at most 15 minutes)
  H1 bad deploy 4512   3 for, 0 against
  H2 database load     0 for, 2 against
  |
  v
Post findings every 5 minutes (code)
  |
  v
Propose a fix: "roll back deploy 4512" (model)
  |
  v
On-call approves --> fix runs --> errors back to normal? (code)

Tools

• query_logs, query_metrics, list_deploys Read-only Code caps their results and saves long outputs to files.
• update_hypotheses(list) Reversible write It edits the list in the incident record.
• propose_fix(action) External It offers fixes from a short approved list, such as a rollback. A separate service runs them after approval.

Another design

• For alerts with one known fix, such as a full disk, a plain script is better than a model.

What could go wrong

• A confident wrong answer misleads the people on call. So the agent shows its evidence and labels guesses as hypotheses.
• A rollback undoes a database change. Each runbook lists its preconditions, and code checks them before offering the fix.

Keep progress in files and git, not in a conversation. Each session starts fresh, reads the progress notes, builds one unfinished feature, and commits when its test passes. Only a passing test marks a feature done.

Session starts and takes the repo lock (code)
  |
  v
Read PROGRESS.md, features.json, git log; run smoke tests
  |
  v
Pick one unfinished feature (model)
  |
  v
Build it --> run its test --> fix --> repeat (model)
  |
  v
Test passes --> commit; code marks the feature passing
  |
  v
Budget nearly used --> write a handoff note, end (code)

Tools

• edit_file Reversible write Each session starts from a clean checkout, so git can restore every file to the last commit.
• run_command Depends on command It runs only in the sandbox, for tests and the dev server.
• ask_team(question) External It posts a question, and the agent moves to another feature while it waits.

Another design

• One long conversation with compaction is simpler, but each compaction can lose a decision.

What could go wrong

• A new feature breaks an old one. So every passing feature's test runs before each commit.
• A crash leaves half-done work. The next session discards uncommitted changes and restarts that feature.

There is no single answer, so grade your design with this table, then with the general rubric below. Many strong designs use a queue with one job per repo. A codemod, which is a script that rewrites code mechanically, makes most of the changes, and an agent fixes the rest.

300 repos --> one job per repo in a queue (code)
  |
  v
In a fresh sandbox: the codemod rewrites the calls (code)
  |
  v
Build and test --> pass --> open or update the PR (code)
  |
  v  fail, or calls the codemod skipped
Agent fixes the rest (model, at most 40 steps)
  |-- pass --> open or update the PR (code)
  |-- fail --> report for a person

Traps in this scenario

• Structured log fields can record whole objects, including personal data that the old format left out.
• Some repos call the logger through wrapper functions, so both the codemod and a text search miss them.