The 40x Inference Gap: Per-Call Routing Just Became a Product Decision, Not an Optimization
The Two-Front War on Your Cost Model
An engineering manager at Uber opened the AI spend dashboard in month four and watched the 2026 budget hit zero. Her engineers had not done anything wrong. They used Claude Code exactly as pitched, at $500–$2,000 per user per month, and the meter ran. Microsoft drew the same conclusion in blunter form: revoke Claude Code licenses across Experiences and Devices, push everyone back to Copilot CLI. That is one side of the squeeze. The other side showed up the same week. MiniMax M3 landed at $0.12 per million input tokens with coding scores reportedly near Opus 4.7. DeepSeek V4, Moonshot's Kimi K2.6, and xAI's Grok Build at $1/$2 per million all shipped in the quarter. The gap between the two sides is 40x.
The unit economics deck written against Q1 pricing now describes a product that doesn't exist anymore. Features killed 18 months ago because the inference math didn't work should be re-evaluated at 50–70% lower costs for non-critical paths.
Why Usage-Based Pricing Is Failing
What teams tell themselves users do: make one thoughtful call, get an answer, move on. What users actually do: chain summarize-then-act, retrieve-then-generate, code-then-test until the context window fills. That is agentic AI with logarithmic value and super-linear costs. Per-user cost grows with tenure because the power users get better at extracting work from the tool. Netflix did not solve this with a cheaper model. They shipped the open-source Headroom proxy to compress redundant tokens between agent code and LLM APIs and recovered $700K. The fix was cutting calls that were not producing value.
The Routing Decision Framework
The error teams keep making is treating model choice as a vendor decision ("we use Anthropic") instead of a per-call decision ("this call routes to X based on the failure mode"). With three Chinese labs shipping competitive models in one quarter, the menu is at least five options wide. A diagnostic you can run on Monday:
- Pull your three highest-volume API calls.
- For each, write down what actually happens when the model gets it wrong.
- If the failure mode is "user retries," route it to the $0.12 tier.
- If the failure mode is "contract breaks or customer churns," keep it at $5 and stop apologizing for the bill.
Anthropic's new Effort Control dial is the vendor admitting the split exists. Teams will package "Quick Answer" against "Deep Research" and price them separately. That is a pricing surface, not an infrastructure feature.
The Capex Signal Says This Isn't a Blip
SoftBank committed €75B for 5GW of French AI data center capacity. Nvidia's Vera Rubin supply chain is running 2x larger than Grace Blackwell. Goldman projects $1T annual AI capex by 2027. Industry capex of $157.7B per quarter against $25B in revenue is a 6.3:1 ratio, which means inference is subsidized today. Features that work at current pricing and break at 2.5x are landmines on a known timer. Features that miss the ROI bar today will clear it at 50–70% lower cost, but only for teams that architected the routing decision before they needed it.
What to do
Map your top 10 AI API calls by volume and classify each by failure-mode severity (retry vs. churn vs. contract breach) to determine routing tier
Evaluate Netflix's Headroom proxy for your agent architecture and run a cost projection comparing current spend vs. compressed redundant tokens
Stress-test AI feature unit economics at 2.5x current inference pricing to identify which features go underwater when subsidies end
Implement per-agent-run cost ceilings and circuit breakers before expanding any agentic feature beyond pilot