We ran four models released in the past week against the same 40 agent tasks, using one harness, three attempts per task, and median scoring. The quality spread was 11 points. The bill spread was eleven times. Both numbers matter, and they point at the same conclusion: routing beats picking.

The setup

Forty tasks in four groups of ten: repository bug fixes with a test suite, document extraction from scanned filings, SQL generation against a 40-table schema, and web research with citation requirements. Every task has a pass condition that can be checked mechanically, which is the only way to score agents without fooling yourself.

Models: GPT-6.2 Astra at $8 and $40 per million tokens, GPT-6.2 Sol at $2 and $10, Qwen4-Max with cache at ¥0.5 per million, and Tencent Hy4flash at ¥0.5 and ¥2 with cache hits at ¥0.02. Costs are computed from actual token counts including retries.

The results

Astra passed 37 of 40 at $1.84 per completed task. Sol passed 35 at $0.31. Qwen4-Max passed 35 at $0.42. Hy4flash passed 32 at $0.17.

Two things stand out. Sol matched Qwen4-Max on pass rate at lower cost, which is not what the price sheets suggest until you count retries — Qwen4-Max needed more attempts on the SQL group, and every attempt is billed. And Astra's advantage is concentrated: it was the only model to pass the two hardest repository tasks, both of which required reading a 900-line file and changing three functions without breaking the other tests.

Where the cheap models broke

Not on knowledge. All four knew the libraries. The failures were structural: tool chains longer than six steps, where a cheaper model repeats a failed command instead of changing approach; ambiguous specifications, where it guesses instead of inspecting; and any task requiring a constraint to be held across more than 30,000 tokens of context.

Hy4flash's five failures were all in the repository group and all the same shape — it fixed the reported bug and broke a neighbouring test. That is a verification problem, not an intelligence problem, and it is partly fixable by giving the model the test suite before it edits.

The routing result

We then built the obvious router: send every task to Hy4flash, check the output mechanically, escalate to Astra only on failure. Pass rate came out at 38 of 40 — one better than Astra alone — at $0.61 per task, a 67% saving.

A 70/30 variant, where anything touching more than two files goes straight to Astra, reached the same pass rate at $0.58 with lower tail latency. That is the configuration we would ship.

Caveats worth stating

One week, 40 tasks, one harness. Variance on individual tasks is high enough that a two-point gap between Sol and Qwen4-Max is not meaningful. Cache behaviour differed between providers and we did not normalise it, which flatters whoever has the better cache implementation. And prices move: two of these four changed cache pricing within the past month.

What survives the caveats is the shape of the answer. The expensive model is worth its price on a minority of tasks, and knowing which minority is worth more than the model choice itself.