Skip to content

    Free chapter

    The One-Agent Baseline

    You decided to split the work across agents. Before you ship the team, prove it beats one agent given the same budget.

    Prerequisites: The multi-agent chapters of the previous book, for what a second agent is and why one might add it.

    The Problem

    A contract-review feature has been built as a team. A planner cuts each contract into parts, reviewers read one part each, a summarizer merges their findings, and a checker reads the result. In the demo it catches clauses the old single-prompt version missed. The team ships.

    Two months later someone asks the question nobody asked at the start: what would one agent have done with the same resources? The old version was never tuned. It ran with a short prompt, a small allowance and no second pass. Here is what the team does on every contract.

    Step 1: Planner splits the contract into four parts
    Step 2: Four reviewers each read their part and the planner's brief
    Step 3: Each reviewer drafts findings, then rereads them to check them
    Step 4: Summarizer reads all four drafts and writes one memo
    Step 5: Checker reads the memo and all four drafts again, asks for a rewrite
    Step 6: Summarizer rereads everything and rewrites the memo

    Count how often the same text is read. The team reads the contract, and then reads its own drafts of the contract, several times over. Nobody ever gave the single agent that many reads, that many chances to reconsider, or that much room to think. The demo compared a rested team with an agent that was set up to lose.

    So the team’s gain has no explanation yet. Maybe the split earned it. Maybe the team simply got more work done per contract, and a lone agent allowed the same work would have found the same clauses. Nobody knows, and the feature now costs several times what it did. The comparison below shows the unfair test next to the fair one.

    An unequal test where a large team is compared with a lone agent holding a small allowance, next to a fair test where both hold the same allowance

    The Core Idea

    Every team needs a baseline: one agent, with the same total budget, the same tools and the same eval set, run next to the team. The comparison of the two is the control run. Until you have it, you have a demo, not a result.

    Medicine solved this problem long ago. A new drug is not tested by giving it to patients and watching them recover, because many patients recover anyway. A trial splits similar patients into two groups. One gets the drug. The other, the control group, gets the best standard care that exists today. Both groups are measured on the same outcome, over the same period, by the same rules. Only a difference between the groups can be credited to the drug.

    Two details of a trial carry over directly. First, the control group gets real care, not neglect. If the comparison group is treated badly, any drug looks good. A lone agent with a lazy prompt is a neglected control. Second, the groups must differ in one thing only. If the drug group also gets extra nurse visits and longer checkups, nobody can tell whether the drug or the attention helped. Extra tokens are the attention. A team that reads, drafts and rereads more is a team that was given more care, and the control has to be given the same.

    A trial with two groups, one receiving the treatment and one receiving standard care, both measured by the same scale

    A team that wins the comparison by spending more has proved that more effort helps. It has not proved that splitting helps.

    How It Works

    The control run has five steps. None needs special tooling. The diagram below shows the shape: one eval set, two arms, one shared budget, one comparison.

    One eval set feeds a team and a single agent that draw on the same budget, and only then are their results compared

    The first step is the one most teams get wrong.

    Fix the Budget in One Unit

    Pick the unit you will hold equal and write it down before you run anything. The usual choice is total tokens across every call in the run: the reasoning, the reading, the output, summed over all agents including the planner and the checker. Count every call. A team’s cost hides in the places that are easy to skip: the setup each agent loads again, the summaries passed between agents, the retries. If you only count the orchestrator, the team looks cheaper than it is.

    Pin two more limits next to tokens: the number of tool calls, and a wall-clock cap. Chapter 2 measures the whole bill. Here you only need the budgets to match.

    Build the Strongest One-Agent Version

    This is the step teams skip, and it is the one that decides the outcome. Treat the control as a product you want to win. Give the single agent the same tools, the same instructions about the task and the same budget. Then let it spend that budget the way a single agent can. It can think for longer before acting. It can reread its own draft and check it against the source. It can run the task twice and compare the results. Each of these is a way to turn spare tokens into quality without a second agent.

    Stop tuning when you would be happy to ship it. If you tuned the team for two weeks and the control for an afternoon, you ran a rigged trial. Give them comparable effort.

    Freeze the Eval Set and the Success Rule

    Use the same eval set for both arms, cases with known right answers or a fixed rubric (Chapter 21 builds team evals in depth). Write the success rule before the first run, so the result cannot bend the rule. Hold some cases back that neither arm was tuned on. A team that was shaped around its test cases wins them for the wrong reason.

    Run Both, More Than Once

    Run each arm on every case several times. A model gives a different answer from one run to the next, and agents give different paths. One good run of the team against one bad run of the control tells you about luck. Compare the pass rate over many runs, and look at how much each arm moves from run to run. If the gap between the arms is smaller than the spread within each arm, you have not found a difference.

    Read the Result

    Compare the pass rates at equal budget. Then decide what to do. There are three outcomes.

    Algorithm Sketch: Control Run
    
    1. Choose the budget unit: total tokens across all agents, plus a tool-call and a time cap
    2. Write the success rule and freeze the eval set, with some cases held back
    3. Build the strongest one-agent version that fits the budget and tune it as hard as the team
    4. Run the team and the single agent on every case, several times each
    5. Total the tokens used by every agent in the team, including retries and summaries
    6. Compare pass rates, and compare the gap with the run-to-run spread inside each arm
    7. Keep the team, fold it back into one agent, or change its shape

    Why a Team Often Wins on Compute Alone

    The reason this control matters is the compute confound: a team nearly always spends more total tokens than the agent it is compared against, and more thinking is itself a way to get better answers. A multi-agent system gets many more reads of the problem, more drafts and more checks. The gain is real, and it is easy to credit to the structure.

    Early evidence supports the worry, with limits you should know. One 2026 study compared single agents and teams on multi-hop reasoning, where an answer needs several linked facts. When both were held to the same reasoning-token budget, the single agent matched or beat the team across the model families it tried. The same study found that some of the reported team gains came from how budgets were set and measured, not from the team. A separate lab write-up, on a browsing benchmark, found that token use alone explained most of the variance in performance.

    Read those carefully. The equal-token finding comes from multi-hop reasoning tasks and from specific model families tested in early 2026. It has not been shown for long-horizon work that leans on many tool calls. So you cannot assume it holds for your task, and you cannot assume it fails. That is the reason to run your own control instead of borrowing someone’s result.

    The Margin Shrinks as the Agent Gets Stronger

    Treat the baseline as a moving target. A team helps most when one agent is weak at the task. When the single agent is already strong, there is less left for the team to add, and the cost of coordination is still there. Beyond some level of single-agent strength, the extra agents stop paying and start to hurt.

    A large controlled study across many team shapes, tasks and model families found this pattern: gains for the team where the single agent did poorly, and losses once the single agent was already doing well. The exact crossover point is specific to the models and tasks in that study. It will move with every model generation, and it is not a number to copy into your plan. The direction is what carries over.

    The diagram below plots the margin against the strength of the single agent. Chapter 3 applies the same idea across model upgrades.

    As the single agent gets stronger, the team’s advantage shrinks, crosses zero and then turns into a loss

    This single-agent threshold has a practical meaning. A team built last year to patch a weak model may sit above its own threshold today without anyone noticing. You can only find out by running the control again.

    Tasks That Split Badly

    The control gives you a result for your task. It helps to know in advance which tasks tend to lose it.

    Sequential tasks split worst. When step four needs the exact result of step three, there is nothing to run in parallel, and every hand-off between agents is a place where detail is lost. A debugging session is the clearest case: each next look depends on what the last one found. Splitting it across agents gives you the same chain of steps plus the cost of passing a summary at each link. In the large study above, planning tasks with strict order were where teams did worst.

    Tool-heavy tasks come next. When the work is mostly calling tools and reacting to what they return, the state lives in the tool results. Dividing the tools among agents means each agent sees only part of the picture, and the agent that needs a result may be a different one from the agent that fetched it. That study also found coordination overhead grew with the number of tools involved.

    Tasks that split well look different. They break into pieces that do not depend on each other, and each piece needs a lot of reading that would crowd one context window. Searching many sources at once is the standard case. The previous book covered why a split can pay, so this chapter stops there. Sequential and tool-heavy work is where your control is most likely to favor one agent.

    A chain of steps where each depends on the one before, next to a set of independent pieces that can be worked at the same time

    What to Do With the Result

    Three outcomes are possible, and each has a next move.

    • The team wins at equal budget, by more than the spread. Keep it, and write the margin down. You will need that number the next time the model changes.
    • The control matches or beats the team. Fold the team back into one agent. Keep what you learned from the team, such as the better instructions or the extra check step, and move it into the single agent’s loop. Chapter 3 covers the fold-back.
    • The team wins in some places and loses in others. Change the shape. Keep the split only where the pieces are independent. Part II covers the shapes.

    What Breaks

    Every control run can fail in ways that make it look like a result.

    A Control That Was Never Tuned

    You will see a single agent with a short prompt and a small allowance losing to a team that had weeks of care. This is the neglected control group from the Core Idea. Give the single agent comparable tuning effort and the full budget, then repeat the comparison.

    A Budget Counted Only in Part

    You will see a team that claims to run at the same budget as the control, with a bill that says otherwise. The usual cause is counting the orchestrator’s tokens and not the workers’, or skipping retries and summaries. Total every call from every agent from the trace, and treat the trace as the source of truth.

    A Difference Inside the Noise

    You will see a gap of a few percentage points on a handful of cases, read as a win. With runs that vary, a small eval set cannot tell a team from an agent. Add cases and runs until the gap is larger than the spread inside each arm, or say the result is undecided.

    An Eval Set That Cannot Tell Them Apart

    You will see both arms passing almost everything. The cases are too easy, so the control run says nothing about the hard ones where a team might help. Add the hardest real inputs you have, and weight them.

    A Result That Went Stale

    You will see a team that won once and was never retested, while the model underneath changed twice. The margin from the first control is a measurement of a model you no longer run. Re-run the control at each model change.

    What It Costs

    The control run costs one extra arm of your eval: a single agent run over the same cases, as many times as you ran the team. That is usually less than the team’s own arm, because the single agent uses fewer tokens per case, though you should check that against your own bill. The larger cost is your own time. Building a strong control takes real effort, and a weak one makes the whole exercise worthless.

    Skipping the control costs more, quietly. A team that does not need to exist keeps running slower and harder to debug every day, and no number says it should go.

    Factor Impact
    Latency Low: the arms run side by side, and the control is the faster of the two
    Tokens and money Low: one extra arm of an eval, and a cheaper arm at that
    Complexity Low: no new system, only a second arm and a fixed budget unit
    Operational risk Low: it runs offline, and the main risk is a control built too weak to count

    In Your Coding Agent

    You can run this control on your own work today. Pick a task you would normally hand to helpers, such as adding a feature that touches several parts of a project. First do it in a single session. Give it a token allowance and a time limit and write both down. When it finishes, record whether the result works: the tests pass, the behavior is right, the diff is something you would merge.

    Then do the same task again with helpers, under the same total allowance and the same time. The allowance has to cover every helper session together, not each one on its own. Merge their results, run the same checks, and compare. Look at what you had to do by hand to make the merged result work, because that effort belongs to the team’s side of the ledger. Try this on a sequential task such as chasing a bug, and on a task made of separate parts such as updating several unrelated modules. Do not expect the same answer from both.

    The Verdict

    Use it when:

    • You have already decided to split and are about to ship or fund a team.
    • A team’s results look better than a single agent’s, and the team spends more tokens.
    • The model underneath has changed since the team was last compared with one agent.

    Overkill when:

    • The task is a one-off that no one will run again.
    • You have not yet built a working single agent, so there is nothing to compare against.

    Try first: Build the one-agent version at the full budget and put it on the eval set before you design any team. If it passes, you are done. If it fails in a way you can name, such as a context too long to hold or tools that confuse each other, that failure is your reason to split, and your control run measures whether the split fixed it.


    Key Takeaways

    • Before shipping a team, run a control: one strong agent, the same total token budget, tools and eval set, and compare.
    • Count tokens from every agent, including retries and summaries, or the team will look cheaper than it is.
    • Many team gains come from extra compute, and equal-token evidence is early and comes from multi-hop reasoning, not from long-horizon tool use, so measure your own task.
    • The team’s margin shrinks as the single agent gets stronger, so a team that won once needs the control again at each model upgrade.
    • Sequential and tool-heavy tasks split worst, and independent pieces that each need a lot of reading split best.
    • Read the result as keep the team, fold it back into one agent, or change the shape.