Ask a team running AI agents which step costs them the most money. In my experience most cannot answer, and the ones who guess are usually wrong.
That is a strange situation. These are teams who will argue for an hour about which model to standardize on, based on benchmark scores from tasks that resemble nothing in their pipeline, while having no idea whether the spend is concentrated in one loop or spread evenly across two hundred calls. The model debate is the fun part. The measurement is the part that would actually change the answer.
This is about the measurement.
Why per-task cost tells you nothing
The number most teams have is total spend divided by tasks completed. Call it $0.40 a memo. It is a real number and it is nearly useless, because it cannot tell you which of the two very different worlds you are living in.
In the first world, your agent makes 200 calls per task, all of them cheap and routine, and the cost is the sheer volume of clerical work. The fix there is fewer calls: better retrieval, less re-reading, tighter loops.
In the second world, your agent makes 200 calls per task, 190 are trivial, and 10 are enormous long-context reasoning passes that account for most of the bill. The fix there is routing: keep the 10, make the 190 cheap.
Same $0.40. Opposite fixes. If you optimize for the wrong world you will spend a sprint and move nothing.
What to log
Four fields per model call. Not per task, per call.
Step name. Not the prompt, the role the call plays in your pipeline. classify_document, extract_claims, reconcile_conflict, final_review. If you cannot name the step, that is your first finding.
Model actually used. Not the model you configured. The one that answered. These diverge more than people expect, and I will come back to why.
Input and output tokens. Separately, always. Output tokens usually cost several times what input tokens cost, so a step that reads a lot and writes a little has a completely different cost profile from one that does the reverse, even at identical total token counts. Anthropic lists Claude Fable 5 at $10 per million input tokens and $50 per million output tokens, a five to one ratio. Averaging those two together throws away the thing you need to know.
Attempt number. Retries are where budgets go to die and nobody logs them, because each individual retry looks cheap and the failure is usually silent. Ours ran about two and a half times my estimate. I have not met anyone whose guess was high.
Four fields. Roughly a day of work. It will tell you more than every model comparison you have read this year.
What the data usually shows
The shape I keep seeing, and the one I would bet on before looking at your logs: spend is concentrated in a small number of steps, and those steps are not the ones anyone would have nominated.
In our research pipeline the distribution was lopsided in a way that made the earlier debate look silly. About 4% of calls were doing work where model quality changed the answer. The other 96% were classification, extraction, formatting, and retries, where a cheaper model returns the same thing. We had been paying a premium tier rate across all of it because that was the model in the prototype and nobody revisited the choice.
Once you can see that split, the routing decision is not a judgment call any more. It is arithmetic.
The rule that survived contact with production
We now assign a model to a step based on one question, asked at the step rather than at the pipeline level:
If this step returns a mediocre answer, how far does it travel before someone catches it?
If the answer is that it stops right there, the cheap model is correct. A misclassified document gets caught by the next step or by a human scanning a list.
If the answer is that it propagates into the deliverable, pay for reasoning. This is the case people underestimate, because weaker models on judgment steps do not produce garbage. They produce something plausible, confidently, and it survives review.
Anthropic’s framing for Fable 5 supports the split, and it is worth reading past the headline claim. Alongside describing it as a Mythos-class model whose capabilities exceed anything the company has made generally available, Anthropic says the model’s lead grows as tasks become longer and more complex. That conditional is the whole argument for routing. If the advantage were flat across task types, you would buy the best model and stop thinking about it.
The tooling matters here more than the discipline. The reason teams do not route is that a model gets chosen in a config file in week one, when correctness is the only concern, and the cost never resurfaces except as a monthly aggregate with no line items. We moved to a stack where the tier is a visible per-step decision. Buda’s approach to cost-aware model routing for AI agents puts the credit multipliers at the point of choice rather than in a pricing page you read once: Sonnet 4.6 at 1.0x, Opus 4.8 at 1.7x, Fable 5 at 3.3x, with Fable 5 offered as a subscription-only premium tier instead of the default. A cost you can see while deciding is a cost you decide about.
Log which model actually answered
Back to that second field, because it is the one people skip and the one that will cost you a day.
Anthropic says Fable 5 includes safety classifiers, and that some cybersecurity, biology, chemistry, and distillation-related requests may automatically receive a response from Claude Opus 4.8 instead. Anthropic puts this at under 5% of sessions on average.
Under 5% is invisible until you are running a security-adjacent workflow and someone files a ticket about inconsistent output. We spent an afternoon on a bug that did not exist. If your logs record the configured model rather than the responding one, you cannot tell a model substitution from a regression, and no amount of staring at the prompt will help.
Start here
Instrument the four fields. Run for a week. Look at spend by step name, then by attempt number.
You will probably find that the expensive question is a small identifiable minority you have been funding out of inertia, and that your retry rate is worse than you think. Both are fixable in an afternoon once you can see them. Neither is visible from a benchmark chart.
