Skip to content
MCP Five

The CrewProject #428 / 51

The ceiling you can't prompt your way out of

240 jars, 0 of 36 found, 439k tokens. A failing agent is expensive precisely because it fails slowly.

Project #3's agent handles one cookie jar beautifully. Project #4 gives it a pantry and finds out where it stops working.

This page is the measurement. It is worth reading even if you never build a sub-agent, because the mechanism it uncovers is a property of every agent loop, including yours.

Part 1 Β· The exam

Sixty jars. Each needs its own inspection β€” one jar per tool call β€” and each report is about 250 tokens of seal state, weights, notes and a movement log.

Nine have been tampered with. The rule for spotting one has two clauses and an exception, so it cannot be pattern-matched from the jar id.

The job: find every tampered jar, and empty only those.

At sixty jars, it just did it

Perfectly. Four iterations, every jar inspected, all nine found, nothing innocent touched.

The honest move is to stop asserting where the ceiling is and go and find it. Make the pantry size a variable, keep the exam identical at every size β€” the same rule, the same decoys, repeating every sixty jars β€” and turn it up.

bash
PANTRY_JARS=240 npm run db:init
PANTRY_JARS=240 npm run dev
PANTRY_JARS=240 npm run compare

Part 2 Β· At 240 jars, here is the ceiling

One agent at 240 jars
Not wrong. Not crashed. Not stuck. It ran out of room, and it charged full price on the way there.
0 of 36tampered jars found180 of 240 inspected
439ktokens spent finding them
36 of 36found by a crew, same job
248ktokens the crew spentcorrect AND ~0.56Γ— the price

Notice what kind of failure that is

It is not wrong β€” it did not identify the wrong jars. It is not crashed. It is not stuck in a loop. Every individual tool call succeeded.

It ran out of room, and it charged full price on the way.

A failing agent is expensive precisely because it fails slowly.

A crash is cheap: it happens immediately and you know. This ran for ten iterations, did real work, produced a plausible-looking partial trace, and delivered nothing β€” having spent more than the successful run would have cost.

Part 3 Β· Why, and it is not the reason people give

The intuitive story is "240 things is too many for the model to think about." That is not what happened, and the real mechanism matters more.

Go back to the meter. Every iteration re-sends the entire conversation. And by iteration 8, that conversation contains every inspection report the agent has read so far.

IterationJars inspected so farRoughly what gets re-sent
2205,000 tokens of reports
46015,000
612030,000
818045,000

Add those up rather than reading them individually β€” because you pay each row, not the last one. That is where 439,000 goes.

Which is why you cannot fix it with a better prompt

There is no wording that makes an agent stop re-sending its own conversation. It is not a prompt problem. It is a shape problem β€” one pair of hands, one head, everything in sequence, and a bill that compounds.

Raising MAX_ITERATIONS does not fix it either. It buys linear capacity at quadratic cost, which is exactly the wrong trade.

Part 4 Β· The kitchen

A head chef does not cook faster. They do something structurally different: split the work, hand pieces to other cooks, and keep only the plan in their own head. The details live with whoever is doing that piece.

Three workers each carry 80 reports and finish. The orchestrator only ever sees three paragraphs β€” and it brings all 36 destructive calls to a single approval.

A single context window makes you pay for everything you have already read, over and over. Splitting the work is how you stop re-reading.

Part 5 Β· How solid are these numbers?

Stated plainly, because the honest version of this table is more useful than the tidy one β€” and because project #4 has already been burned once by a confidently-wrong figure.

RowConfidence
60 jars, accuracySolid. Three attempts per case, both modes, 100% in both β€” 6/6 one agent, 12/12 crew.
60 jars, costThe softest number here. Two attempts of the one case both modes run identically, and the run-to-run spread (85k–99k single, 74k–83k crew) is a meaningful fraction of the gap. "A wash" is as strong a claim as this data supports.
240 jarsA single attempt per mode, plus a standalone probe that independently agreed. The direction is not in doubt β€” one mode finished the job and the other emptied nothing β€” but treat the magnitudes as one significant figure.

The compare run and the standalone probe give slightly different totals β€” 440,517 / 245,948 versus 439k / 248k. Both are real; they are different runs. This site quotes the probe figures because those are what the source README's headline table uses.

Part 6 Β· The crossover is a property of your job

Not of agents in general.

Whether delegation helps depends on three things, all of which are about your workload rather than about the technique:

FactorPushes crossover earlierPushes it later
size of each tool resultlarge (250-token reports)small (a number)
number of calls the job needsmanyfew
iteration captightgenerous

What you now know

  • The premise was wrong at 60 jars, and finding that out on stage 5 was the cheapest possible outcome.
  • At 240 jars a single agent found 0 of 36 and spent 439k tokens β€” not crashed, just out of room at full price.
  • The mechanism is re-reading: cost grows quadratically while capacity stays fixed, so the two curves cross.
  • You cannot prompt your way out of it, and raising the iteration cap buys linear capacity at quadratic cost.
  • A crew wins structurally β€” fresh context and a fresh budget per worker β€” not because more agents are smarter.
  • The crossover is a property of your job. Measure it; do not assume it.