Do You Even Need an Agent?

Pick the lowest rung of autonomy that solves the task, let the model choose the next step only when your code cannot know it in advance, and check the choice again when the model changes.

Prerequisites: None

The Problem

Finance wants to know why a taxi ride cost eight hundred dollars. The expense was filed by the receipts agent, approved by a manager who trusted it, and paid. You open the trace, the step-by-step record of a run:

Step 1: Agent reads the receipt photo -> taxi, total 800, in yen
Step 2: Agent calls the filing tool -> amount 800, category travel
Step 3: Agent answers -> "Filed your taxi receipt for 800."

The feature is simple. An employee photographs a receipt, and an expense line lands in the reimbursement system, in dollars. The team built it as an agent: a language model (the model, from here on) with two tools, convert currency and file expense, and a prompt that lists the steps. On most runs the model follows them. On this one it skipped the conversion, and eight hundred yen, the price of a short ride, went out as eight hundred dollars.

The prompt has been patched twice. It now says to convert every amount, even one that looks like dollars, and to call the filing tool exactly once. Read the patches together and a pattern shows: each one says what happens next. The team knew the exact path before the first receipt arrived, wrote it out in English, and then paid a model to read it back on every run and, usually, follow it.

The fix is one model call and a few lines of code. The call reads the photo into fields, and code converts the amount at the day’s rate and files the line. The model still reads every receipt, but it decides nothing, so no step can be skipped. The comparison below shows both designs.

Nothing in the agent version was badly built. The mistake came earlier, when the team handed the model a decision that had only one right answer.

The Core Idea

Every system that uses a model sits on a ladder of autonomy, and each rung up hands one more decision from your code to the model. This book uses one definition throughout: an agent is a system in which the model decides the next step. Below that rung, your code decides what happens next and the model only fills in content.

Food reaches you in four ways that map onto the ladder rung for rung. A vending machine is a single model call: you press B4, the B4 snack drops, and a wrong snack is obvious the moment it lands. A set menu is a fixed workflow. Each course takes real skill, the way each model call does real work, but the kitchen fixed the order and the price before you sat down.

A restaurant with three set menus, where the host asks one question at the door, is a router. Meat, fish or vegetarian: one decision, made early, then a sequence the kitchen cooks every night.

A personal chef is an agent. You say: something light, no nuts, six people at eight. The chef opens your fridge, decides what to cook, shops for what is missing, and tastes along the way. No set menu can serve a request that open. You also cannot know the dish or the bill in advance, and you cannot test a chef the way you test a vending machine. Hire a chef when the fridge is a mystery and the guests are particular, never to hand out snacks. The illustration below lines up all four, from least freedom to most.

Every decision you hand to the model is one you can no longer predict before the run. Hand over only the decisions your code cannot make.

How It Works

The autonomy ladder has four rungs: a single model call, a fixed workflow, a router and an agent. The system gets harder to predict with every rung you climb. The ladder below marks who decides what happens next on each one.

Rung One: A Single Model Call

One input goes in, one output comes out, and your code does everything around it. Reading a receipt into fields is one call, and so is drafting a reply for a person to edit. It is also the easiest rung to test: collect real inputs, write down the right outputs, and compare.

Below this rung sits plain code. If a lookup table or a regular expression does the job, no model belongs in it. The currency conversion in the opening is that kind of step.

Rung Two: A Fixed Workflow

A fixed workflow chains several model calls in an order your code sets. The model fills in each step, and your code chooses the steps. A contract review might pull the obligations out of the contract, check each against policy, and draft a memo.

Because the order is fixed, you can test each step on its own, a failure points at one step, and the cost of a task is a number you can write down before launch. Chapter 3 covers the workflow patterns.

Rung Three: A Router

A router hands the model exactly one decision about control: which of several paths to take. The model reads the input and returns a label from a closed list you wrote. Your code maps the label to a handler, which is fixed code or a fixed workflow. After the label, the model decides nothing.

This is the most underrated rung. Plenty of requests that sound like agent work, such as “handle whatever arrives in the finance inbox”, turn out to be a handful of known paths behind one unknown: what kind of request this is. A router answers that one question and leaves the rest as code you already test. Like the host with three set menus, it asks the only open question once, at the door. And you can measure it like a classifier: label real messages, run the router over them, and count.

Ask for the label as structured output, a reply format your code fixes in advance: here, one field that can hold only a name from your list (Chapter 4). The model can no longer invent a route, but it can still pick the wrong one, so you need a measure of doubt. Some interfaces return the score the model gave each option, a confidence at no extra cost. Many return only the answer. Then give doubt a route of its own, an “unclear” label, and where a wrong route is expensive, run the router twice and treat disagreement as doubt. A confidence the model writes into its reply is only more generated text.

Whichever signal you use must be honest. A router is calibrated when its confidence means what it says: of the messages it routes at 90 percent confidence, about nine in ten land in the right lane. A calibrated router that is wrong three times in ten is still useful, because it tells you which three. Route automatically above a threshold and send the rest to a person. Check calibration on your own labeled messages before trusting the number, and again after any change of model, prompt or routes. Study the most confident mistakes first: your code will act on those without hesitating.

Algorithm Sketch: One-Step Router

1. List the routes, each with a distinct name, a one-line description and a fixed handler
2. Add an "unclear" route that hands the message to a person
3. Give the model the message and the list, and constrain its reply to one route name
4. Read a confidence: option scores if the interface returns them, or agreement between two runs
5. If the route is "unclear" or the confidence is below the threshold, send it to a person
6. Otherwise run that route's handler, with no further model decisions
7. Record each message, route and confidence, and re-check accuracy after every change

Rung Four: An Agent

On the top rung, the model decides the next step, then the one after it, until it decides the task is finished. Your code gives it tools, meaning actions it can request such as looking up an order, then runs whichever one the model picks and hands back the result. Chapter 2 opens up the loop that does this.

An agent earns its place when the path depends on what earlier steps found. Working out why last night’s data import dropped rows might take one query or six. You count the source rows, read the error file, check yesterday’s schema change, and stop wherever the cause turns up. Nobody can write that sequence down in advance, because the sequence is the answer.

The price is everything the lower rungs promised. Two runs on the same input can take different paths, and when a run fails, the step that went wrong may be one you never planned.

Reaching this rung no longer means writing the loop. A general-purpose harness, the kind your coding agent runs on, already runs it and gives the model a shell and files to work in. You specialize it with your instructions, tools and skills, packaged instructions the agent loads only when a task calls for them (Chapter 10). Your application keeps the interface, the context and the approvals, and the harness keeps the loop. Build a bare loop once to understand it (Chapter 2), and your own harness only when no existing one can do the work (Chapter 27).

Three Questions That Pick the Rung

Three questions place any task on the ladder. Ask them in order.

Is the path known in advance? If you can write the steps down before you see the input, write them in code, as a single call or a fixed workflow. If there are a few known paths and the only unknown is which one applies, use a router. A task needs an agent only when each next step depends on what the last one revealed.

How costly is a wrong step? An agent will take wrong steps, so price them before you build. Reading the wrong record costs a second. Paying the wrong person is hard to undo. Let the model explore where mistakes are cheap, and put the expensive actions behind human approval, a gate in your code, since a prompt rule only lowers the odds (Chapter 15).

Can you check the output? An agent you cannot check is an agent you cannot improve. A test suite, or a total that must reconcile with the ledger, separates a good run from a merely plausible one. If the only judge is a person reading every run, you pay for the person and the agent. Narrow the task until something can check it, or stay lower on the ladder.

Worked Example: One Finance Team, Three Rungs

The finance team wants three jobs automated, and each belongs on a different rung.

Task	Rung
Turn each receipt photo into an expense line	Single call
Sort each email in the finance inbox into status, policy, card or mismatch	Router
Explain why a card statement does not match the expenses filed against it	Agent

The receipt job is the opening’s fix. The inbox job has four known lanes and one unknown: which lane. A status question gets a lookup and a templated reply, and a policy question gets the matching section of the policy. The router makes that one decision, and three of the four lanes stay code the team already tests. A wrong route costs the minute it takes someone to forward the email.

The mismatch job is the only one with an unknown path. A gap between the statement and the filed expenses can come from a refund posted the next month, a charge split across two receipts or a receipt filed twice, and which record to read next depends on what the last one showed. Wrong reads are cheap and a wrong pay deduction is expensive, so the agent may read any record, but anything that moves money waits for a person. The output is checkable: with its explanation applied, the totals must reconcile.

Put together, every receipt takes the one-call path, the router picks each email’s lane, and only the mismatch lane hands off to an agent. Rungs compose: the agent goes in the one lane that needs it, with cheap, tested code everywhere else. The diagram below shows the inbox.

The Line Moves

The three questions are answered against the model you have. With each model generation, more open-ended work fits inside one loop. Teams that rebuilt hand-built structures on newer models, such as a classifier in front of a fixed pipeline or specialist agents behind a router, have often found that one agent with good tools and skills did better, and sometimes cost less. The structure was doing work the model no longer needed done for it.

So read this chapter’s lean toward the lower rungs as a rule for known paths. A router whose lanes are fixed code keeps its place, while a router in front of specialist agents is the kind of structure that gets folded into one. Fixed code also stays in charge of the triggers that start a run, such as a schedule or an incoming message, of the gates where a check or a person must approve, and of high-volume paths, where every extra call is multiplied by every input.

Re-run the rung decision at each model upgrade: keep a set of real cases with known right answers (Chapter 12), try the simpler design on the new model, and keep whichever wins. A stronger model might let the finance team’s mismatch agent take policy questions too, with the policy loaded as a skill. The receipt path stays one call and code.

What Breaks

Ladder failures come in two kinds: standing on the wrong rung, and standing on the right rung without the check that keeps it safe.

An Agent on a Known Path

You will see a prompt that keeps growing numbered steps and “never do this” rules, and runs that follow the same sequence nearly every time. Each rule is code written in English and enforced by hope. Move the sequence into code.

A Structure the Model Outgrew

You will see a model upgrade that changes little, because a classifier and a fixed pipeline still make the decisions the new model could make better. Rebuild the open part as one agent with tools and skills, test both on the same real cases, and keep whichever wins.

A Router With Overlapping Routes

You will see the same kind of message land in different lanes on different days, its confidence split between the same two routes. A better prompt will not fix a bad route list. Merge the overlapping routes, or redefine each one by the action its lane takes.

An Agent Nobody Can Check

You will see runs that look fine, then hear from a customer that they were wrong. Without a check outside the model, every prompt change is a guess. Chapter 12 shows how to build that check.

What It Costs

Count the calls in the opening example. The fixed path makes one model call per receipt. The agent makes three, one to ask for the conversion, one to ask for the filing and one to write the answer, and each re-reads the photo and the run so far. And three is the good run.

The money is the part you can see. The larger cost is that an agent’s behavior varies from run to run. You test a workflow step by step. You test an agent by running each case several times and judging the outcomes, because one good run proves little. The personal chef bills by the hour, and the hours depend on what the chef finds in the fridge. The chart below shows many runs of the same task.

Here is the bill for the top rung, against a fixed workflow doing the same job.

Factor	Impact
Latency	High: calls run one after another, and their number varies per run
Tokens and money	High: deciding calls come on top of the work, each re-reading the run
Complexity	Medium: you need step limits, traces and outcome checks a fixed path never needed
Operational risk	High: the model can take actions in an order nobody tested

In Your Coding Agent

Your coding agent is the clearest case for the top rung, and it passes all three questions. Ask it to fix a failing test and the path is unknown. The cause might sit in the test, the code or a configuration file three folders away, and which file to open next depends on what the last one showed. Wrong steps are cheap, because an edit can be reverted. The output is checkable: the tests pass or they do not, however confident the model sounded.

Now ask it to format a file. The path is known, and a formatter already does it the same way every time. That task is not on the ladder at all. It is plain code. An agent that formats by deciding, run after run, to call the formatter spends model calls on a decision with one right answer, and one day it will skip it. A fixed step that runs the formatter after every edit needs no model, and Chapter 3 shows where such steps belong.

The Verdict

Use it when:

• The next step depends on what the last one found, so nobody can write the path down in advance.
• Something outside the model can tell a good run from a bad one, such as a test suite.
• Wrong steps are cheap, or the expensive ones wait for human approval.

Overkill when:

• You can write the steps down. A prompt of numbered instructions is a workflow paying agent prices.
• Every request is one of a few known kinds, which a router with fixed handlers covers.
• The only way to judge the output is a person reading every run.
• The user is waiting, and a fixed path would answer in one or two calls.

Try first: The lowest rung that could work: one call, then a fixed workflow, then a router with a fallback to a person. Move up when the fallback lane fills with requests that need several lookups, or when your steps start to depend on what an earlier step found. At the top, configure an existing harness before you build one, and re-run the choice at every model upgrade.

---

Key Takeaways

• An agent is a system in which the model decides the next step; on every lower rung, your code decides.
• Start at the bottom of the ladder (single call, fixed workflow, router, agent) and stop at the first rung that works.
• The router is the most underrated rung: one constrained decision, tested code behind each route, and doubtful cases sent to a person.
• An agent needs an unknown path, wrong steps that are cheap or gated, and a check outside the model, and it belongs only in the lane that needs it.
• The line moves with each model generation: re-run the rung decision at every upgrade, and keep fixed code for triggers, approval gates and high-volume paths.