Your finance team can tell you what you spent on AI last quarter, broken out by model, by team, by application, by input tokens and output tokens and cache reads. Mine can too. What neither of us can pull from that invoice is which half of the spend produced work anyone actually used, and which half produced attempts, retries, and output that a human quietly rewrote before shipping it.
AI cost optimization fails when it targets the wrong unit. The rate card prices consumption, a benchmark prices a task, and only the third number, the cost of an accepted result, is what your business actually paid.
Your tooling is fine. The gap sits in what the source data can express, which is why so much cost advice lands with a thud.
One number shows the shape of it. On the public Intelligence Index maintained by Artificial Analysis (v4.1, retrieved 28 July 2026), Anthropic's Opus 5 carries half the per-token price of Claude Fable 5 and finishes only about a quarter cheaper per completed task, roughly $2.03 against $2.75. Half the rate card, a quarter of the saving. The difference is consumption, and consumption isn't printed on anybody's price list.
Your invoice is an allocation report, not an attribution report
Let me concede something first, because the alternative is condescending to people who have already done this work. If you run a serious engineering organization in 2026, you probably are not flying blind on AI spend. You have it broken out by model and by team. You can chargeback. You may have a FinOps practice that is genuinely good at this, and the discipline they brought over from cloud is real.
That practice answers one question well: who spent, on what resource. The question waiting for you in the budget review is a different one. What did the spending accomplish?
Those two questions need different source data, and only one of them has any. An invoice has a field for the model. It has a field for token counts. It has no field for this call recovered from a failed attempt, or this call independently checked another agent's work, or this output was rejected and rewritten by hand on Thursday. You cannot aggregate your way to information the underlying records never captured. Better allocation produces better allocation.
A Sequoia partner recently described where we've landed as a token-rationalization moment. Right instinct, slightly wrong frame. Rationalization implies we bought too many tokens. We bought them without knowing which ones did work worth having.
The rate card is only one of two multiplicands
Even at the level everyone assumes is settled, the intuition misleads.
Price per token is not what a task costs. Token consumption varies substantially between models, so the real figure is price per token multiplied by tokens consumed to finish the job. Both terms move. Only the product is real. Artificial Analysis, which runs every model through the same Intelligence Index and publishes output-token consumption alongside dollar cost, has made this checkable for anyone who wants to click through: their index v4.1, retrieved 28 July 2026.
The comparison I opened with is the mild version. Go back a generation and it gets stranger. Fable 5 costs twice as much per token as Opus 4.8, but running the whole index came to about $6.2K against Opus 4.8's $3.7K. That lands under 1.7x rather than the 2x the rate card advertises, since the pricier model needed less of what it was charging for. Those two totals come from an earlier index run rather than the current v4.1 numbers above, so read them as an illustration of the mechanism rather than a live comparison. Against that pairing the premium model was the more token-efficient one, though Artificial Analysis still counts Fable 5's output as verbose next to the field median.
Two cautions.
First, these figures describe max-effort, adaptive-reasoning configurations, and the Fable 5 numbers include fallback behavior. Reasoning effort is itself a dial that moves consumption enormously. The numbers describe a configuration, not a model in the abstract.
Second, none of this makes expensive models secretly cheap, and none of it says anything about open weights versus closed. License category predicts nothing about token efficiency in either direction. If you take one thing from the arithmetic, take the shape of it rather than any particular row: LLM cost per task is a product of two numbers your procurement conversation probably only discusses one of.
The US government's AI standards evaluators found the same effect independently. Measuring cost per correctly solved task rather than cost per token, they watched a single open-weight model swing from meaningfully cheaper to meaningfully more expensive than its comparator, GPT-5.4 mini, depending on which benchmark it was solving. Same model. Same rate card. Different work.
What a benchmark task still leaves out
From here the argument stops being arithmetic and starts being about how you build things.
A benchmark task cost is clean because a benchmark is clean. One prompt, one attempt, one scored answer. Your workflow does something else entirely. Before anyone uses an output, your system may have made three attempts at it, replayed the same context four times, called a specialist model for one subproblem, invoked a deterministic tool because the model kept getting arithmetic wrong, run a test suite, had a second model review the result, and then had a senior engineer spend twenty minutes fixing what survived all of that.
Every one of those steps consumed budget. None of them appears in a per-task benchmark figure, and the invoice records them as undifferentiated consumption.
This is the claim the rest of this piece rests on: two workflows with identical model spend can produce materially different amounts of accepted work, and the gap is decided mostly by how the work is assigned, scoped, and verified rather than by which model was purchased.
If that is false, the correct strategy really is procurement. Buy the cheapest adequate model, cap the spend, stop thinking about it. I don't believe it's false, and neither, apparently, do the people building the platforms. But it is falsifiable in your own environment, which is the point. The reflex when the numbers disappoint is to decide the model isn't good enough. Usually the model was fine and the composition around it wasn't.
The direction runs both ways, which is why this isn't a defense of premium models. A stronger model can lower total workflow cost by diagnosing a problem correctly on the first pass and handing a bounded implementation task to a cheaper lane, avoiding the failure loop entirely. A cheaper model with a tight specification and a good test can beat a premium model given a vague one. Neither is a rule. The composition decides, and you design a composition rather than buy one.
Illustrative proportions only. The segments carry no measured values.
One line item, many different jobs
Ask what an AI agent cost you last month and you'll get a figure that is technically correct and analytically useless. Model consumption does different kinds of work, and the differences are economic rather than descriptive.
Some consumption advises: it produces a plan, a diagnosis, a routing decision that shapes everything downstream. Some executes a bounded task. Some explores a question nobody assigned, which occasionally turns out to be the most valuable spend of the quarter. Some coordinates other agents. Some recovers from a failure that shouldn't have happened. Some writes to memory. Some exists purely to check whether other consumption produced something true.
These carry different value and different risk. They are billed identically.
That observation is not mine originally. Angela Jiang, who works on Anthropic's platform, put it plainly on Sequoia's Training Data in July 2026: "if tokens aren't really fungible and you need to give them different jobs, like maybe this token is advising versus this token is executing." She was describing a coordination layer they're building, which means a frontier lab now treats job assignment as roadmap rather than as an implementation detail. Worth noticing where the interesting problems moved.
Your invoice knows none of this. Neither does a spend cap, which is the part that should worry you.
Where standard AI cost optimization advice breaks down
When the bill spikes, most teams reach for cheaper models, prompt caching, or a spending cap. Each is sound under conditions. Each encodes an assumption about your workload that the aggregate bill prevents you from checking.
Caching assumes your reuse interval fits inside the cache lifetime and that the write premium is recovered by hit rate. Your traffic pattern decides that, not caching itself.
Cheaper models assume consumption holds roughly constant across the swap, which the two-multiplicand problem says it does not.
The cap is the one I'd argue with hardest, and one of Anthropic's platform leads made the case better than I would on the same podcast, warning about what happens when you are "just like, here's a cap and you're stuck within your cap." A cap treats all consumption as equally cuttable. Under pressure it doesn't trim proportionally. It cuts whatever lacks an immediate deliverable, which means exploration and verification go first, precisely the two categories whose absence you won't notice until something ships wrong. The same conversation offered something about caching that a vendor had no obligation to say: it's worth doing, and the bigger gains may sit a layer above it, in how model work gets composed in the first place.
What we paid to learn
A company describing its internal tooling can slide into sounding like a product announcement. This isn't one. It is what a recent stretch of AI-assisted engineering work in our company cost us in lessons.
Caching is a bet on your access pattern. On a recent project in our company we introduced caching into a document-oriented workflow where the arithmetic looked obvious. Measured against the traffic the product actually had, it came out meaningfully more expensive, across more than one operation type. We rolled it back. The lesson is narrower than the quotable version: caching is not a scam, it frequently saves a great deal of money, and I would reach for it again. It's a wager on reuse behavior, cache lifetime, and write premium. Apply it to an aggregate bill and you never find out whether the wager paid.
Assurance has an economic envelope, and it can be exceeded. On another stretch of work we ran a deliberately verification-heavy delivery workflow: separate planning, plan review, implementation, and code review, each on a different model lane, with retries tracked. The output quality was genuinely good. Manual review afterward found very little to correct . It also consumed an envelope out of proportion to the consequence of the tasks it was applied to, and it contributed to us running into our own limits sooner than we'd planned. Both halves are true and I'd rather print both than the flattering one. Verification isn't waste. Assurance is a purchase, and we bought a premium tier for work that didn't need one.
The third lesson arrived from outside. An external reviewer going through one of our client runtimes found it had been sending the full system-prompt stack twice on every turn. Not a subtle bug, and not one anybody inside had caught, because it never surfaced as an error. It surfaced as slightly more consumption per call than anyone had a reason to question. We fixed it. What stayed with me was where it turned up: inside a system built specifically to manage context. An always-on tax can survive in exactly the place most likely to be looking for it, because nobody instruments the thing they already believe they've solved.
The seam where descriptions like this inflate is the gap between primitive and strategy, so: we have implemented specific primitives: model tiers, bounded specialist contexts, structured returns instead of whole transcripts, role separation, delegation depth and fan-out limits , and child budgets debited from a parent. Those exist. The higher-order routing strategy that decides what goes where still lives in operating rules, prompts, and skills, not in a supervisor that makes those calls on its own. There's no shipped product behind this section. There's a set of seams I can point at.
Verification is a purchase with a dial
Separating creation from evaluation is worth real money when stakes justify it, and the mechanism is simple: an independent checker doesn't inherit the creator's assumptions.
The checker usually doesn't need the creator's entire conversation either. Handing over the full transcript feels thorough and mostly buys you tokens. What the checker needs is the claim, the evidence, and the acceptance criteria.
The part most teams get wrong is treating review depth as a constant. Consequence varies enormously across task classes. A reversible internal draft and an irreversible customer-facing commitment do not warrant the same assurance spend, and setting one depth for both means overpaying on most of your volume to protect a minority of it. Depth is a dial. Set it per task class, by consequence, uncertainty, and how expensive the mistake is to undo.
The number almost nobody can compute
Everyone agrees, once you say it out loud, that the unit should be cost per accepted result. Almost nobody can calculate it, and the reason sits upstream of any tooling: you cannot measure acceptance without a definition of done.
Research on how people actually use these systems keeps landing near the same place. A University of Texas at Austin and KPMG study spent eight months inside one firm's back-office operations, working through roughly 1.4 million interactions to separate the people getting real value from AI from the much larger group simply using it. Discussing that work afterward, the researcher who led it noted how rarely people in the system set any acceptance criteria or boundaries, or asked the model to check its own output. Plenty of organizations will be further along than that. It still describes a condition common enough to plan around.
Which produces the connection I keep coming back to. The bill being one undifferentiated number and the acceptance criteria being missing are the same problem wearing two faces, one turned toward finance and one toward engineering. You cannot attribute spend to value when nobody has written down what counts as value.
What to write down on Monday
None of this requires buying anything, which is deliberate. The minimum viable instrumentation is a ledger you can keep in whatever you already use, recorded per run: the task class and what happens if it's wrong, which model and role handled it, what context got loaded, how many attempts it took, whether it escalated, how much human correction time it absorbed, what the verification cost, whether the result was accepted, and who accepted it.
Two weeks of that on a single workflow, not the whole estate, gives you an attribution picture no provider invoice can generate. It also tells you where you're actually spending, which is rarely where people guess.
Orchestration can turn into an expensive machine that justifies its own existence. Every agent you add brings calls, context, latency, new failure modes, and maintenance nobody scheduled. For a great many tasks, one capable model with a clear prompt and a real test will beat a composed workflow on cost and reliability both. Take this as a warning against orchestration by default. It earns its cost; it doesn't get a presumption of value, and the right target is the least complicated composition that reliably crosses your acceptance threshold. Sometimes that composition is one model and a test, and applying this argument correctly means arriving there on purpose rather than by accident.
If your AI spend is genuinely small, the instrumentation may cost more than it saves. Measure when the number starts mattering, not before.
Better assignment, not indiscriminate austerity
The reflex when a bill grows is to cut. Badly aimed, usually, because cutting an aggregate cuts capability and consumption in whatever ratio the aggregate happens to contain.
Solvimon put the underlying idea well: rate cards are the price of the ingredient, not the meal. The durable advantage has little to do with access to the cheapest model or the smartest one. It comes from knowing what kind of intelligence a piece of work requires, what evidence makes its output acceptable, and what that accepted result actually cost you.
That knowledge lives in your workflows, your acceptance criteria, and your records, which is to say it lives with you rather than with any provider. Keep the governance, the memory, the permissions , and the definition of done portable, and let the model layer be the part you can change your mind about.
