We Built Five Multi-Agent Architectures and Gave Them the Same Job: One Agent Won, the Handoff Team Never Saved a File

We Built Five Multi-Agent Architectures and Gave Them the Same Job: One Agent Won, the Handoff Team Never Saved a File#

A multi-agent system splits one job across several AI agents: one for shipping, one for returns, one for membership, each looking only at its own part. It sounds like it should be more accurate than one agent juggling everything. We built five ways of splitting an agent and gave all five the same job. The best result came from a single agent that loads documents when it needs them: 48 of 48 questions right, every hidden test passed on the coding task, and the fewest tokens of any pattern that got full marks.

The worst result came from the team that passes the conversation along. On the coding task, four agents handed the work to each other until the last one, who had never seen the shipping price table, made the prices up ("Zone 1: $5"; the real document says $4.00) and ran out of tokens before saving anything. Hidden tests passed: 0 of 60, in all three runs.

What "splitting an agent" means#

An agent is a model in a loop: it reads the conversation, either answers or calls a tool, sees the tool's result, and goes again. The question is what to do when the job needs more knowledge than you want in one prompt. In our test, the knowledge is three policy documents for a made-up online shop: shipping, returns and membership.

LangChain's post Choosing the Right Multi-Agent Architecture (January 14, 2026) names four patterns. We added the plain single agent as a baseline. Here are all five, described by what a customer question goes through:

• Single: one agent with all three documents in its system prompt and no tools.
• Skills: one agent with a load_skill(name) tool. It starts with a short list of what each document covers and pulls in the ones it needs.
• Subagents: a supervisor with no documents. Its tools ask_shipping, ask_returns and ask_membership each start a fresh specialist that sees only its own document and the request the supervisor wrote.
• Handoffs: a front desk plus three specialists sharing one conversation. transfer_to(agent) swaps whose system prompt is active; the conversation history stays.
• Router: one call classifies the question, the chosen specialists answer it in parallel, and one last call combines their answers.

We used no framework. All five patterns live in one Python file, so every model call is visible. The difference between skills and handoffs, for example, is a few lines. In skills, the tool result is the document, so it stays in the conversation:

for tc in msg["tool_calls"]:
    name = args_of(tc).get("name")
    out = DOCS.get(name, f"unknown skill {name}")
    msgs.append({"role": "tool", "tool_call_id": tc.get("id", ""), "content": out})

In handoffs, the document belongs to the specialist's system prompt. A transfer only changes who is speaking; the tool result is the line "transferred to shipping":

nxt = args_of(tc).get("agent")
convo.append({"role": "tool", "tool_call_id": tc.get("id", ""),
              "content": f"transferred to {nxt}" if ok else f"unknown agent {nxt}"})
if ok:
    state["active"] = nxt

So when the shipping specialist hands over to returns, the shipping price table leaves with it. That detail decides the coding round.

Why we measured it ourselves#

LangChain's post compares the patterns in a table of model calls and tokens. For a question that spans several domains, it lists subagents at about 9K tokens and skills at about 15K, and says subagents process fewer tokens overall. The numbers carry a "~" and rest on one stated assumption, that each agent's documents are about 2,000 tokens. The post does not say they came from runs, and as a way to reason about cost, an estimate like that is fair. We wanted to see what happens when the five patterns actually run on the same questions.

The test: a fake shop and a calculator#

We wrote three policy documents for Kestrel Supply, a shop that does not exist: shipping (1,188 tokens), returns (898) and membership (756). Every number is invented, so the model cannot answer from memory. The right answers come from a calculator that implements the policies in code, so no person grades anything.

One sentence matters later. Like many real company documents, the shipping policy says who owns what: "Membership discounts are NOT described here; they are owned by the Membership team." We put it there on purpose.

The settings were the same for all five: Qwen3.5-27B Q4_K_M on llama.cpp, temperature 0, thinking off, each task run three times (seeds 42, 43, 44). Because temperature is 0, the three runs came out almost identical. They show the results are not luck; they are not three independent samples.

There were two rounds:

1. Questions: 16 questions, 48 runs per pattern. An answer counts if its last line ANSWER: <number> is within half a cent of the calculator. Answers cut off at the 2,048-token limit count as wrong.
2. Coding: write three functions (shipping_fee, delivery_days, refund) in one file, kestrel.py. The agent can run 5 public tests. We grade the final file with 60 hidden tests.

Round 1: questions#

Questions that need only one document were easy: all five patterns got 12 of 12. The patterns separate when an answer needs two or three documents.

The other 12 runs per pattern were two-turn questions (a follow-up in the same conversation), and every pattern got the second answer right in all 12.

The clearest failure is a two-document question: "A Silver member ships an 8 kg parcel by Standard service to a Zone 5 postcode that starts with 98. What is the total shipping fee?" The answer is $28.40: an $18.00 base fee with the 20% Silver discount, plus a $14.00 remote-area surcharge.

In handoffs, the front desk correctly transfers to shipping. The shipping specialist then reads the ownership sentence, quotes it, and keeps the question anyway:

"The policy explicitly states in the introduction: 'Membership discounts are NOT described here; they are owned by the Membership team.' Since I am the shipping specialist and my policy document does not contain rules for Silver member discounts, I cannot apply any discount."

It answers $32.00, all three times. Its system prompt told it to transfer when the case needs a policy it does not have, and it had just written that it lacked one. On the same question, skills loaded shipping and membership in its first call and got $28.40.

This was a pattern, not one question. Handoffs got 21 of 48 runs wrong, and in none of those 21 did the membership specialist ever receive the conversation. Subagents failed in a similar way on one three-document question: the supervisor asked shipping and returns and never asked membership.

Router failed differently. On a Gold-member electronics return, the returns specialist said the item was past its window and the refund was $0.00, while the membership specialist said it could not compute the amount without the returns rules. The combining call went back and forth between the two ("However" six times, "Wait" three times in the recorded run) until it hit the token limit without an answer. That happened on the same question in all three runs. Across the round, router was cut off in 11 of its 48 runs, sometimes at the combining step and sometimes at a specialist.

Round 2: coding#

The coding task is harder for split teams in a specific way. Membership rules change both the shipping fee and the refund, so whoever writes the membership part has to edit code someone else wrote.

The single agent read the file, wrote it once, ran the tests and stopped. Subagents and router also got every hidden test right, because one agent (the supervisor, or the merge step) collected the specialists' code and wrote the file. Getting there cost them about twice the tokens: 43,002 and 45,738 against 21,601 for skills.

Handoffs is the scene from the top of this post. Here is the relay, with how many tokens each agent wrote. The quotes come from a rerun with the first run's seed, which reproduced its calls and token counts; the scores come from the measured runs only.

1. Front desk (27, then 94 tokens): reads the file, has no documents, transfers to shipping. That part is right.
2. Shipping (56 tokens): writes no code. "Let me also get the returns and membership policies by transferring to those specialists." It transfers to returns, and its own price table goes with it.
3. Returns (49 tokens): the same. "Let me also get the membership policy." Transfers to membership.
4. Membership (4,096 tokens, cut off): starts with "I have all three policies." A few lines later it writes that it does not have the shipping and returns text in its context. It actually holds one document, membership.

The membership specialist's prompt says "never guess a number," and it copies that phrase back into its own reasoning five times. Then it writes "Shipping Policy Assumptions (typical values)" and fills in numbers:

None match. It hit the 4,096-token limit while writing, and in none of the three runs did a file get saved. The same order of transfers (front desk, shipping, returns, membership) happened all three times.

Our tool description may have helped cause this. transfer_to says "The next agent sees the whole conversation and the same file." The documents live only in each agent's system prompt and do not travel, but the shipping and returns specialists' words ("let me also get the ... policy") read as if they thought a transfer would bring the policy back to them. We did not run a version with different wording, so we cannot say how much of the zero comes from that sentence.

Put the two rounds side by side and handoffs failed in opposite directions. With questions, specialists did not hand over when they should have. With code, they did nothing but hand over.

How this compares to LangChain's table#

LangChain's estimate has subagents using fewer tokens than skills on a multi-domain question. In our runs it went the other way: subagents used 1.46 times the tokens of skills on questions (6,422 against 4,391) and 1.99 times on coding (43,002 against 21,601).

Our guess is document size. Our documents are 756 to 1,188 tokens, smaller than the 2,000 the post assumes. When documents are small, loading one costs less than the messages agents write to each other: the supervisor's request, the specialist's answer, and the supervisor reading it back. With larger documents the order may flip back. We did not run larger documents, so this stays a guess.

The post also cites Anthropic's result that a multi-agent research system (Opus 4 leading Sonnet 4 subagents) beat a single Opus 4 by 90.2% on its internal research evaluation. That is a different kind of job: searching widely in parallel, where separate clean contexts help. Our tasks were about combining rules from three documents into one number or one file, and our results speak only to that.

What we suggest#

Start with one agent, even when the task looks hard. One agent can do more than you might expect. When the documents get too big to keep in one prompt, let that agent load what it needs: skills got every question and every test right here, with the fewest tokens of any pattern that got full marks. Split into a team only when there is a reason one agent cannot handle, such as work that really runs in parallel.

If you do build a team, decide first who writes the final file and who gives the final answer. The subagents and router teams got the coding task right because one agent was in charge of the file. The handoff team had no such agent, and it ended with a made-up price table and nothing saved.

We used the same shop-support setting as our earlier run of Anthropic's single-agent-loop kit, where folding a router and six subagents into one loop also finished more tasks (18 of 19 against 16). If you run these five patterns on another model, we would like to hear what changed. New measurements go out in the newsletter below.