The agent loop

Observe, decide, act, repeat

Robots and control systems have long been described by one cycle: observe the world, decide what to do, act, then look at the result. LLM agents run the same cycle.

Here is what each word means in an agent:

• Observe: your code puts the current situation into the context, including the result of the last action.
• Decide: the model reads the context and replies, either with an answer or with a request for an action.
• Act: your code checks the request and carries it out.
• Observe the result: your code adds what happened to the history.
• Repeat: a loop in your code goes around again.

Only the decide step happens inside the model. Some models write out their reasoning before they decide, but that reasoning is part of the same reply. Between calls, nothing is thinking. Your code is just running.

The same loop in engineering terms

Framework docs use many names for the same five pieces. This table says what each one actually is.

Step through a run

The widget below lets you step through three runs, one step at a time. At each step, watch who is acting, what your code remembers, and what the model sees. Notice that the model's view goes empty between calls, because the model is not running then.

Three things in the widget come up again later in the course:

• The context gets longer with every call, because your code sends the whole history each time.
• Each tool result carries the id of the request it answers, so the model can match results to requests.
• In the dangerous-action run, the rule that blocks the request lives in your code, not in the prompt. A person decides, and the run continues from a saved checkpoint.

How a run ends

Say the agent edits a file and replies, "Done, the test passes now." Should the run end? Usually yes, but only after your code checks that the test really passes. A good agent has several ways to stop, and you choose them on purpose:

• The model says it is done. It replies without asking for a tool, or it calls a special "finish" tool.
• A limit runs out. Your code caps the number of steps, the tokens, the cost, or the time.
• A check passes or fails. For example, your code runs the tests. Some agents refuse to accept "done" until the check passes, and reply to the model with "The tests still fail. Keep going or explain why."
• Errors pile up. The same tool keeps failing, or the model repeats the same request, so your code stops or asks a person.
• A person stops it. The user cancels, or rejects an action.

Here is a sketch of those rules in code:

def should_stop(state, reply):
    if state.user_cancelled:             return "stopped by the user"
    if state.step >= state.max_steps:    return "step limit reached"
    if state.cost >= state.max_cost:     return "cost limit reached"
    if state.repeated_calls(window=3):   return "stuck: repeating itself"
    if not reply.tool_calls:                  # the model says it is done
        if state.success_check():             # confirm with evidence
            return "success"
        state.add_note("Check failed: " + state.check_output)
    return None                               # keep going

Notice the order. The hard limits come first, so a model that keeps saying "done" while the check fails still cannot run forever. Then a reply without a tool request leads to a check, not straight to success. That is the split from chapter 0: the model says it is done, your code stops, and a check confirms it.

In the sketch, success_check() is a few lines of your own code that look at the result itself, not at what the model said about it. For Patch, it runs the tests and passes only if they all pass. The model may say it is done several times in one run, and the check runs each time. When the check fails, your code adds the failure to the history, and the loop goes on. The model reads the failure, keeps working, and later says it is done again. Only a passing check ends the run as a success.

When no test can check the result

Many tasks have no test. Say you ask an agent to summarize a 40-page contract for a manager. Part of "good" is a judgment call, but you still need a check. So split "done" into smaller claims, and check each one with the cheapest thing that can judge it:

• Write down what done means, before the run. Turn the vague goal into a short checklist. For example, the summary names both parties, lists every payment and deadline, says how either side can end the contract, and fits on one page. You can write the list once for each kind of task, or the agent can draft one for the user to approve. If the request is too vague for a list, the agent should ask the user first.
• Check in code what code can check. Code can confirm that the summary fits on one page, has every section, and quotes only text that really appears in the contract. These checks cannot prove the summary is good. But they catch the usual early stops, such as a skipped section or a made-up quote.
• Ask a separate model call to grade the rest. Send it the contract, the summary, and the checklist. Ask for pass or fail on each item, with a quote from the contract as evidence. Use a fresh call instead of asking the agent, because agents tend to approve their own work (chapter 15). This grader makes mistakes too, so compare it with a person's judgment on some runs before you rely on it.
• Let a person decide when taste decides. Then the run ends with "ready for review" instead of "done". The person accepts the result or sends it back with notes. Make the review quick by showing each checklist item next to its evidence.

The rule is the same as for Patch. The model's "done" starts the check, and evidence ends the run. Only the checker changes. Chapter 16 compares code, model, and human checkers in more detail.

A few words, defined

Docs use these words loosely. In this course they mean:

Example: why long runs get expensive

Your code sends the whole history on every call, so the total cost grows faster than the number of steps. Suppose the fixed part of the context is 3,000 tokens, and each step adds 1,000 more.

call 1 sends   3,000 tokens
call 2 sends   4,000 tokens
call 3 sends   5,000 tokens
...
call 20 sends 22,000 tokens

total over 20 calls:
  20 x 3,000 + 1,000 x (0 + 1 + ... + 19)
= 60,000 + 190,000
= 250,000 tokens

So twenty steps cost a quarter of a million tokens, and doubling the steps roughly quadruples the cost. Prompt caching makes the repeated part cheaper, but the context still fills up. Chapter 4 shows what to do about that.

Other shapes of the loop

The basic loop has a few common variations. They differ in who decides the next step and in how the work is split up. This is a first look; chapter 14 covers them in depth.

Simple loop

This is the loop from the widget, often called a ReAct loop. The model picks every step. Start here. It handles a wide range of tasks when the tools and the stopping rules are good.

Event-driven agent

Here the loop runs once for each event, such as a new chat message, a webhook, or a scheduled time. The agent loads its saved state, runs the loop, and saves the state again.

def on_event(event):
    state = store.load(event.thread_id)    # what we knew before
    state.add(event)
    run_loop(state)                        # the same loop as before
    store.save(state)                      # keep it for the next event

This fits agents that live for days and react to the world, such as support bots, inbox assistants, and scheduled jobs.

Graph or state machine

Here you draw the process as boxes and arrows. Some boxes call the model, and some arrows depend on a check or on the model's choice. For example: research, then draft, then review, then either revise or publish.

This fits when you already know the shape of the process. You get a natural place to save progress after each box, and a natural place for a person to approve.

Planner and executor

One model call writes a plan, and a loop carries out the steps. If a step fails, the planner revises the plan. This helps with long tasks where steps depend on each other, or when a person should approve the plan first. Chapter 6 covers when planning helps.

Parent agent with subagents

One of the parent's tools starts a second agent loop with a fresh context. The second loop does a focused job and returns a short result. This helps with independent research in parallel, and with keeping messy exploration out of the parent's context. Chapter 7 covers the costs.

Who decides what happens next?

Each shape gives a different answer to this question. At one end, your code decides every step and the model fills in blanks: that is a workflow. At the other end, the model decides every step within a few hard limits: that is an open agent.

Put each decision where the knowledge is. If you know the process, write it in code or as a graph. If the next step depends on what the agent discovers, as when debugging, let the model choose. Hard limits such as budgets, permissions, and approvals always stay in your code.

Running example: Patch gets a loop

In version 1, Patch replaces you with a loop. It gets one tool, run_command, so it can run the tests itself, and it stops when the model replies without a tool request or after 30 steps. Patch can now try a fix, run the tests, read the failure, and try again. It can also repeat a bad idea until the step limit stops it.

- suggestion = model([system, user(paste)])          # v0: you were the loop
+ answer = run_agent(task, tools=[run_command], max_steps=30)   # v1

What goes wrong

You decide, AI builds

Check yourself

Takeaways

• Observe, decide, act means: build the context, call the model, run the tool, record the result. Only "decide" happens in the model.
• Give every run several ways to stop: the model's "done" (checked against evidence), limits, error rules, and a person.
• Because the history is sent on every call, cost grows much faster than the number of steps.

Going deeper

• ReAct: Synergizing Reasoning and Acting in Language Models (Yao and others, 2022). The paper behind the simple loop: the model alternates between reasoning and actions, and tool results correct its guesses.
• Building effective agents (Anthropic, December 2024). Covers stopping conditions, checkpoints where a person can step in, and the common loop shapes.