The Scientist

Revenue share is the dangerous half

Of Benedict Evans's two pricing moves, Alibaba's reported consideration of revenue share instead of flat token pricing is the one that breaks a model rather than shifting a number. A per-token rate is a forecastable variable cost: caching, distillation, prompt compression and shorter agent loops all pay back directly. Revenue share ties gross margin to a supplier's take rate and is invariant to every efficiency win the team ships. Reported consideration, not a published price sheet. DeepSeek's increase is so far described only as "significant," with no rates attached.

The second-order effect matters more than either provider. DeepSeek sits on the cost/performance Pareto line, the set of options where nothing else is both cheaper and better, alongside Kimi and Grok. When the cheapest good option raises its price, everything above it gains headroom. Evans calls this a supply crunch in which labs can name their price, and leaves open where price, capacity and capex re-equilibrate.

The frontier claim does not survive a methodology review

"Alibaba and Kimi very close behind US frontier labs on some benchmarks" names no suites, no eval protocol, no contamination controls, no per-task breakdown. Pareto position is workload-dependent, and the asserted line carries no price basis, no context window, no batch-versus-streaming distinction. The same account credits Grok to SpaceX rather than xAI, a provenance slip that is itself the argument for re-deriving the frontier on in-house evals.

A frontier chart never tells you which task mix produced it, and the task mix decides whether the result transfers to your distribution.

Where the open-weight floor actually sits

The counterweight is open weights. A 30B-parameter, agent-loop-trained model is published under Apache 2.0, sized for a single 24GB consumer GPU, weights under 20GB at roughly 4-bit, with a claimed 0.2–1% accuracy loss, per Devshot. Two gaps. The ~233 tok/s figure for DFlash speculative decoding, a drafter proposing 16-token blocks verified in parallel, was measured on an RTX 5090, while "fits on 24GB" is a 4090 number. What that figure doesn't tell you is the draft acceptance rate, batch size or context length, none of which is disclosed. A 16-token block pays off only if most of it survives verification, and acceptance degrades where agents live: long, noisy tool-output context.

Sub-20GB of weights on a 24GB card leaves roughly 4GB for KV cache and activations, so real agent trajectories force an 8-bit KV cache, a hard context cap, or a bigger card. The accuracy claim names no benchmark and no turn depth, and single-shot quantization deltas systematically understate compounding error across a multi-turn loop. None of that makes the release unusable. It makes it a priced ceiling on what any hosted provider can charge you, leverage at renewal whether or not you deploy it.

What dual-sourcing has to mean operationally

Evans reports that large companies now presume they should dual-source, because "you can't rely on anyone right now." Slogan, until a swap executes in under a week. Real means a provider-agnostic harness with per-task golden sets scored jointly on quality and $/successful-task, the only unit that survives both a model swap and a move from per-token pricing to revenue share. It also serves as regulatory insurance: the US is exploring bans on importing Chinese models while China discusses export restrictions, so access can vanish with no notice. Attribution is the third leg. Databricks has published its cost-efficiency methodology, an enterprise AI FinOps category is forming around spend attribution and ROI proof for coding agents, and retrofitting per-feature, per-team, per-trajectory tags across a live agent fleet costs far more than instrumenting them first.

What "seconds to nanoseconds" actually decomposes into

Optiver's retreat system reprices an entire option surface, thousands of contracts for S&P options, after every trade, and The Pragmatic Engineer reports it went from seconds to nanoseconds. That is nine orders of magnitude, and the write-up never decomposes it into algorithm, hardware, and narrowed scope. The mechanism it does name is the transferable one: in the fastest market-making cases the strategy layer is memoized, meaning precomputed responses burned into FPGA or ASIC hardware. This is not distillation or quantization or a clever approximation; the decision surface is enumerated offline and the answers shipped as hardware.

Read the source critically. The claims are company-supplied through a friendly profile, and the model-level evidence is zero: no accuracy numbers, no ablations, no backtest methodology. "Sub-nanosecond" is physically implausible for a full tick-to-trade loop, since light travels roughly a foot per nanosecond, so it can only describe an internal stage or a jitter measurement. The 30–40% platform-engineering share of about 950 engineers, set against a "typical 15–20%," cites no source and never defines platform work. Firms tend to announce that latency has commoditized shortly after their own latency edge has.

The same inversion, in an ordinary stack

Canva's session-revocation rebuild is the portable version, and Devshot's account carries the numbers Optiver's lacks. Hundreds of millions of active sessions moved off MySQL onto Amazon S3 as compact immutable records: 16-byte binary records, a 12-hour window served as 30-minute objects, conditional GETs to fetch only changed chunks, conditional PUTs for optimistic concurrency on merge, and lookups against sorted in-memory arrays. Reported result: roughly 88% less cache footprint, 2,000+ revocations per second through async workers, and the backing database down to two read replicas, because load now scales with write throughput instead of gateway count.

The same pathology sits in most ML serving stacks, where feature flags, embedding index version pointers, model registry pointers, and online blocklists are polled by every serving node. Fixed-width records, time-bucketed objects, and delta fetch invert the scaling relationship. The fix is a week of work, not a migration.

The invoice: freshness becomes a correctness property

Every precomputed answer is a stale answer, and the sources price that bill three ways. At Optiver, staleness is adversarial; the retreat system exists because faster counterparties pick off stale quotes. Any model whose outputs an external party can observe and act on faster than the features refresh has identical exposure: pricing, fraud detection, bidding, dynamic ranking. Canva's 12-hour window in 30-minute buckets makes propagation delay a monitored quantity rather than an assumption.

The quiet version is worse, because nothing pages. A partial or stale restore, features materialized with a six-hour freshness gap or an index rebuilt with a fraction of documents missing, presents downstream as unexplained metric degradation and burns days of debugging misattributed to data drift. Uptime dashboards measure whether the service answered. They do not measure whether the answer used current data. A Rubrik-sponsored resilience agenda gestures at that gap with no RTO or RPO numbers behind it, which makes it a hypothesis for local instrumentation: freshness SLAs plus distribution checks on served features.

Two adjacent details worth stealing

• Timestamp resolution is a correctness property. Optiver contributed a nanosecond-precision timestamp type upstream to Postgres because the precision was not available out of the box. Event logs that tie at millisecond granularity give non-deterministic ordering and non-reproducible training snapshots, and the failure mode looks like offline lift that never materializes in production.
• An external bound beats a better model. Knight Capital's single bug cost $440M, against a backdrop of hundreds of production changes daily. Optiver's control is strict separation of signals, strategy, and execution, with risk systems that can block trades and halt strategies. Any model that actuates a consequential action needs a model-external output bound and a tested kill switch.

Precomputation is the cheapest latency win available, and its price is a freshness guarantee. Averages hide the tail where the money leaves.

Why this result is worse than a null

The finding is not that probes underperform. It is that hidden-activation probes — small classifiers read off a model's internal states to flag buggy code without executing it — produce signals that move depending on how the activations are extracted, and isolating the specific bug does not resolve the inconsistency, per Devshot. That is an architectural result, not a tuning result. Correctness is not sitting in the activations as a stable linear direction waiting to be read out. A probe fit to one extraction recipe on one bug family has no claim on transfer, which means variance across extraction methods is the primary metric, not accuracy under whichever recipe happened to work.

The roadmap line this kills is a popular one: internal model signals as a cheap stand-in for running tests. Execution remains the oracle. If the workstream survives anyway, the honest deliverable is an extraction ablation grid reported with variance, plus a written statement that it supplements the test suite rather than replacing it.

The reliability gap is architectural, not a model upgrade away

A survey of roughly 336 papers concludes that GUI and computer-use agents systematically lack error recovery, safety checks, and auditability. Those are properties of the harness, so a stronger base model does not supply them. The finding sits directly against the capability claims circulating alongside it.

The recall figure deserves a second look, because high recall with unmeasured precision is frequently net-negative. A reviewer that flags everything teaches engineers to dismiss it, and the dismissal habit outlives the tool by a long way. Evans's own calibration line is the right posture for all of it: stop believing these are glorified autocomplete, and stop believing AGI has arrived.

Two things that transferred

First, structural context beat embedding similarity on a causal question. Grounding an LLM investigation in a service map improved first-guess root cause by more than 25 percentage points over rival AI methods. Most ML stacks already own a better graph than an AIOps vendor does: the dbt or feature-lineage DAG plus the model dependency graph. Seed retrieval from the failing node's neighborhood, rank by similarity within that neighborhood, and score the result against the current embedding-only baseline on replayed incidents. The thing the 25-point number does not tell you is how much of it survives a graph that is stale.

Second, the governance shape got validated at the highest available bar. A recent Linux kernel release candidate landed 400+ fixes credited to automated scanning, including an eight-year-old use-after-free race in ptdump that Syzbot flagged in June and an LLM helped root-cause, with human triage and sign-off retained as mandatory. Read the pipeline rather than the headline: a deterministic detector, an LLM explainer, and a human gate. The data-quality analog already has the first component in constraint tests and anomaly monitors. The explainer drafts the root cause and the fix. The human signs. That division of labor is why the kernel result is credible where the self-improving-agent numbers are not.

Ship the pipeline where a deterministic detector finds the fault and a model explains it — not the one where the model grades its own work.