The Scientist

Decompose the spend into four factors

What Uber's engineering team published is an observability schema first and a finance number second. Each factor has its own owner and its own lever: session count sits with product, turns with the agent loop, tokens with prompt construction, price with procurement. A blended dollars-per-request figure is unattributable the moment it moves, because the line goes up and any of six things could have moved it. That schema is worth copying even by teams that reproduce none of the optimizations.

The forecasting use is sharper than the reporting use. Once spend is multiplicative, unit-cost work and demand growth are separate terms compounding against each other, and the elasticity here was brutal: unit cost fell by roughly a third, demand made it back more than nine times over. An efficiency program presented to finance without the absolute-spend series beside it loses the budget argument the following quarter, however good the ratio looks.

Code-mode batching saves tool tokens

Code-mode batching, where the agent emits code that calls tools so results never round-trip through the context window, is credited with saving 55-100% of tool-execution tokens. The width of that range is itself the finding: it signals severe workload heterogeneity, and 100% definitionally means no tool result ever entered context, which holds for one narrow class of workflow. Measure the local distribution rather than adopting the headline, with a task-success guardrail attached, because token savings that depress completion rate are negative ROI wearing an efficiency label.

Progressive disclosure is the companion lever. With 1,000+ MCP tools registered against a 24-million-node, 80-million-edge context graph, tool-schema tokens scale linearly with catalog size, JSON schema competing with real task context inside one window. Token bills don't break out schema boilerplate, and at that catalog size it is most of the prompt; CLI-resolved tool discovery cuts it, while swapping in a cheaper checkpoint leaves it untouched.

Skill dispatch is a ranking problem with known metrics

3,600+ skills against 30,000+ executions per day averages roughly 8.3 executions per skill per day, which in practice is a power law with a long dead tail. The stated authoring rule, that a skill description is a router written in the words a user would actually type, is a top-k retrieval spec with the prose left on. Build a labeled query-to-skill dataset, report precision@1 and precision@3, gate catalog additions on regression. Description collisions arrive with scale, and teams without that eval spend weeks debugging a ranking failure as a model failure.

One number to discount: 70%+ of pull requests being agent-attributed is an attribution metric wearing a productivity metric's clothes, with no revert rate, no defect density, no review latency and no LOC normalization behind it.

Where the surrounding evidence lands

Two corroborations tighten this. The same company's database team documented static rate limiting and static quota management failing outright under hyperscale multi-tenant load before moving to adaptive load management. Most inference gateways still run that identical open-loop control: a fixed QPS cap per tenant on a workload whose cost per request varies by an order of magnitude with prompt and generation length. The fix is closed-loop concurrency control on observed queue delay, shadow-tested before enforcement.

On the platform side, analyst consensus reported by The Information puts Snowflake at $1.481B revenue (+29%) and MongoDB at $734.4M (+24%), with prints landing Wednesday and Tuesday. A consumption vendor compounding at a quarter to a third per year is a compute bill compounding at the same rate against a flat data budget. Salesforce reworking how it charges for AI is the second signal this quarter that per-seat margin models optimize an objective the invoice no longer uses.

Cost per successful task, multiplied by demand, is the only metric that reconciles with the invoice.

Sparse activation cuts compute and leaves memory alone

The efficiency claim in Tencent's release is a FLOPs claim: roughly 49B of 770B parameters activate per token. Memory is indifferent to that. Every expert stays resident, and at FP8 that is about 770 GB of weights, above a single 8xH100-80GB node's 640 GB and a tight fit on 8xH200 at roughly 1.13 TB. So the realistic configuration is two H100 nodes with expert parallelism, roughly $32-48/hr at commodity rates. Beating the hosted $0.834 per million input tokens requires about 10.7K tokens/sec sustained aggregate at $32/hr, or around 16K at $48/hr. High-batch prefill-heavy jobs reach that. Spiky decode-heavy agent loops do not.

The trap sits between the two headline features. Long-context requests consume KV cache across 78 layers, which collapses the concurrent batch, which collapses the throughput the breakeven depends on. The same GPU-hour buys 1M-token context or cheap tokens per unit of work, not both. The published cost model is also incomplete: the output-token price is undisclosed, output typically prices at 3-8x input, and output is the dominant term in agentic workloads.

The win is inside the noise

The persuading number is human preference: 2.99 of 4 across 203 engineering tasks judged by 163 experts, against Kimi K3 at 2.94 and GLM-5.3 at 2.92. No confidence intervals, no inter-rater reliability statistic, no stated paired design. The power arithmetic settles it. Detecting a 0.05 mean difference at 80% power and alpha=0.05 needs roughly 785 paired tasks if the per-task difference SD is 0.5, and about 2,540 if it is 0.9. At n=203, first place and third place are the same place.

The agentic numbers fail differently. Terminal Bench 2.1 at 85.4 and DeepSWE moving 28.0 to 64.3 are scaffold-dominated results. Tool sets, retry policy, max-turn budget and prompt harness move those scores materially, so a score published without the harness spec is not reproducible by anyone outside the lab. A 36-point generational jump on a software-engineering suite is also the magnitude that earns a decontamination audit before belief. The architecture claims ship without ablations. Gated sparse attention is never isolated against dense attention, and the built-in speculative-decoding head reports no draft-token acceptance rate, which is the one number that predicts real decode speedup.

One structural detail carries more information than any score: every model in the comparison set is from a Chinese lab, and no Western closed model is cited as the bar.

Two accounts of one week, opposite failure modes

Trade coverage of the same week's two open-weight drops, Hy4's preview and Z.ai's model, carried no benchmark names, no parameter counts, no context length and no ablations, only "near-frontier capability at superlow costs." The same event is therefore documented twice: once with numbers nobody outside the lab can verify, once with no numbers at all. Those two accounts do not need reconciling. They mean no independent party has measured any of this yet. GLM-5.3's own headline has the same shape, better than Claude Opus 4.8 at under half the output tokens, measured on Z.ai's in-house code bench, where token count conflates verbosity with capability.

That makes the decision procedure cheap. Score candidates on cost per solved task at fixed pass@k on the local harness, with bootstrap intervals on the delta rather than point scores, and record model ID, quantization, backend and sampler config on every eval row so the comparison survives the next release. Then let the routing layer absorb the result, not a migration project.

Free weights are a license, not a serving cost: the parameters stay resident whether the experts fire or not.

The failure concentrates where the citations do

The aggregate rate is not the useful number in NPR's test. On the items where Claude failed to reject a false narrative, it cited state-aligned sources more often than its peers did. Citation composition, conditioned on incorrect answers, is therefore a leading indicator of answer failure, and it points at the retriever rather than the generator. It is also the cheapest auditable artifact you can hand a security reviewer or a public-sector buyer: tag every retrieved and cited domain against a provenance tier list, then report composition on the failure slice separately from the aggregate.

Framing was the second deliberate design variable. Every narrative was posed in a neutral and a leading form. Duplicating existing eval items into presuppositional variants costs a few hours and produces one of the few safety metrics that reliably moves when a reranker changes or a system prompt gets trimmed. The framing delta belongs in a standing regression gate, not a one-off study.

Calibrate power before anyone builds a slide: separating a 75% debunk rate from an 80% one at 80% power takes roughly 1,200 items per arm. Nobody in this reporting ran that, and neither does the median internal safety suite. That is why a 50-prompt set cannot support a claimed three-point gain from a prompt change.

Execution telemetry is structurally blind to behavior

Barnadeep Bhowmik's "cognitive observability" essay names the same gap on the online side. A retrieval endpoint returns 200 OK in 380ms with a complete trace, citing a document that does not exist. The latency histogram is green, the error rate is zero, and detection recall on that failure is zero. Effective time-to-detect equals however long a customer takes to file a ticket. The label is new marketing on an old measurement gap, so use the frame and hold off on buying the category. No detection-rate comparison against existing offline harnesses has been published.

These are complements, not substitutes. Most teams run two of the three, and the missing one is almost always the behavioral layer.

The confounder underneath every latency number you have

Adyen's engineers traced mysterious Kubernetes pod setup timeouts to conntrack garbage collection, kernel connection-tracking table pressure presenting as generic cluster slowness. ML workloads are the archetypal high-churn tenant: Ray, Argo and Kubeflow pods cycle constantly, and data loaders open thousands of short-lived connections to object storage, feature stores and vector databases. A serving-latency benchmark run on a cluster with no conntrack instrumentation carries an unmodeled confounder in its p99, and slow pod startup directly inflates autoscaler response under traffic spikes. Alerting on nf_conntrack_count against nf_conntrack_max above 80%, plus insert-failure counters, is an afternoon of work.

And the eval unit is about to change again

OpenAI has placed agentic coding and enterprise knowledge work under a single product lead, treating both as one primitive: a persistent, stateful coworker. This is an n=1 organizational signal from a paywalled interview with no reliability numbers attached, but the technical consequence is precise. A single-turn accuracy table over (prompt, response) pairs cannot see memory drift, mid-trajectory state corruption, or a tool-retry storm that triples cost while the final answer still reads correct. Trajectory-level metrics are what that architecture requires: multi-session task success, memory precision and recall against ground-truth state, cost per resolved task, state-corruption rate. Almost nobody has them instrumented.

If you measured the chat endpoint and shipped a summarizer, you published a better model than the one your users get.