The Engineer

The 88% is a memory-layout win, not a storage win

Canva's saving comes from representation, not from S3. A fixed-width 16-byte record in a contiguous sorted array deletes the per-entry object headers, pointers and hashmap load-factor slack that dominate heap at hundreds of millions of entries. Lookups are binary searches over sorted in-memory arrays, not hashmap probes. That is why the structure survives the size. It is cache-line friendly and never rehashes. Memory cost is exactly length times sixteen bytes. Writers merge new chunks with conditional PUTs for optimistic concurrency. Gateways refresh with conditional GETs, so only changed 30-minute chunks cross the wire. Async workers absorb 2,000+ revocations per second. The revocation database is down to two read replicas.

The precondition goes unstated. A rolling 12-hour window only bounds memory if the access-token TTL is shorter than that window. Otherwise a revoked credential ages out of the window and silently becomes valid again. That regression throws no error and fires no alert. Long-lived tokens need a separate path. Two questions the teardown leaves open are the ones to settle before copying it: the gateway refresh interval, which is the real revocation SLA, and the behaviour when S3 is degraded. Fail open, fail closed, or serve last-known-good.

Optiver's version: the broker hop as a durability purchase

Optiver built "PG Feed", an internal NOTIFY/LISTEN implemented on the Postgres write-ahead log, to avoid the extra disk reads and writes Kafka adds to high-fanout distribution, per The Pragmatic Engineer's teardown. The same team contributed a nanosecond-precision timestamp type to Postgres, which otherwise tops out at microseconds. That is real craftsmanship, and the generalizable test is one sentence: a broker in the hot path is a purchase of durability and replay semantics. If no consumer replays from an offset and nobody needs multi-day retention, that is an fsync per message for a guarantee nobody exercises.

The failure mode to own on day one: an unconsumed replication slot pins WAL and fills the primary's disk. Without slot-lag alerting and a disk-headroom threshold, this pattern eventually takes down the database it was built to stop touching.

Calibrate the numbers before citing them

Optiver's nanosecond-class path does not compute decisions fast. It does not compute them at all. Strategy responses are memoized and burned into FPGA or ASIC behind a strict signals-to-strategy-to-execution separation, with a "retreat" system repricing thousands of options after a trade so a faster adversary cannot eat stale quotes. "Seconds ten years ago, nanoseconds now" redefines the measured unit of work, from full surface recomputation to per-quote emission out of a hardware lookup. It is not a 10^9 optimization curve, and "sub-nanosecond" is gate-level inside silicon rather than a system boundary. The same discipline belongs in any benchmark published this quarter: name the boundary, the unit of work and the noise floor. Optiver itself now says latency is no longer the moat and invests more in model quality than in shaving microseconds. A firm that measures in nanoseconds calls micro-optimization a floor cost.

A broker in a latency-sensitive path is a subscription to durability. If nobody replays from an offset, cancel it.

Where the tokens actually went

The 48-66% is a context accounting result, not a model improvement. Tool definitions get serialized into the prompt on every turn of the agent loop. Twelve typed JSON schemas is a few thousand tokens of static overhead, multiplied by however many turns the task takes. The second-order effect is bigger: a wide tool surface raises the tool-selection error rate, and every wrong pick burns a full turn plus the retry reasoning. Collapsing twelve tools into one moves the branching out of the model's tool-choice space and into the tool's argument space, where a CLI string costs a handful of tokens instead of a schema. Unadvertised third effect: the invariant prefix gets longer, so the system prompt becomes a better prompt-cache candidate.

The trade-offs land on on-call, not the vendor. Per-operation typed validation disappears. Malformed calls come back as opaque strings from the CLI instead of schema rejections, so the error taxonomy degrades exactly where eval instrumentation needs resolution. It is also a hard dependency on one CLI's argument surface. Pin browser-use 3.0 explicitly. After this refactor, a minor CLI change is a silent behavioural regression in the agent.

Why this lever matters more this quarter than last

Benedict Evans reports DeepSeek has notified users of a forthcoming "significant" price increase. Reuters reports Alibaba is considering asking for a revenue share on model usage. Both sit on the cost/performance Pareto line, the best model available at each point on the cost curve. The cheap end is where prices are rising. Large enterprises Evans talks to now presume dual-sourcing by default: "you can't rely on anyone right now." That turns model portability into a requirement with a measurable bill. Capacity is being locked years ahead through structured finance. The FT describes a $200bn arrangement in which Google guarantees TPUs sold to an SPV that leases to Anthropic on Apollo and Blackstone debt. Rate limits are a downstream artifact of deals like that.

Tokenizers differ between providers, so identical text produces different token counts and therefore different invoices. Naive cost comparisons are invalid. Prompt-caching semantics, structured-output and tool-calling schemas, and reasoning-effort knobs are all provider-specific. Rate-limit shapes differ: tokens per minute versus requests per minute versus concurrency. That changes backpressure design, not a config value. A gateway that papers over it buys a one-config swap and a lowest-common-denominator feature set.

Local inference is a third option, priced honestly

Meta's Muse Glimmer (30B, Apache 2.0) runs on a single 24GB GPU or a 32GB Apple Silicon Mac. Roughly 4-bit quantization puts weights under 20GB for 0.2-1% accuracy loss. DFlash speculative decoding proposes 16-token draft blocks verified in parallel, roughly tripling throughput to about 233 tok/sec on an RTX 5090. Speculative decoding is distribution-preserving, so that 3x is genuinely free. Good craft. It is not 3x agent wall-clock: multi-turn loops are dominated by prefill and tool-call round trips, so prefix and KV caching come first. The capability claim ships with no benchmark name and no score. Shadow-deploy it and score cost per successful task and tool-call format adherence across multi-turn state. A local model that emits malformed calls 5% of the time burns the savings in retries.

Cost per token is the provider's number. Cost per successful task is the only one you own.

What was actually automated, and what was not

Not review, and not the merge decision. Syzbot fuzzing detects, an LLM (Claude Opus 4.8) explains causality, and a named human signs off. That pipeline closed an eight-year-old use-after-free race in ptdump that Syzbot flagged back in June. It landed in a release Devshot reports shipped on August 9 with 400+ fixes credited to automated scanning. Stable is expected around August 16.

This transfers because the expensive half is already built. Sanitizers and fuzzers already run in your pipeline, and the untriaged crash corpus already exists and grows faster than anyone reads it. Triage backlogs die at the explanation step. Turning a sanitizer trace into a causal story about ownership and object lifetime is the slow part, and it is exactly where a model is demonstrably useful, because ground truth here is a reproducer rather than a judgment call. The output is verifiable by execution. That property is what separates this from the agent claims that do not hold up, and it is why the cheapest AI win available this quarter is probably not a new platform.

The same cost curve runs the other way

Anthropic shipped what is described as a large public cryptanalysis tool. Alex Stamos frames the dynamic as identical to the encryption debate: the tools that let defenders find problems are freely available to attackers, and the threshold for an attack drops. The operational translation is unglamorous. The dominant security metric shifts from prevention to mean-time-to-patch, because the cost of discovering a bug in a dependency tree or a public API surface is falling monthly on both sides of the fence. Two consequences carry quarter-scale lead times. Point coding agents at your own code and dependency graph before someone else does, and complete a crypto primitive inventory with a documented rotation path. Crypto agility takes quarters to retrofit and cannot be bought during an incident. OpenAI's GPT-5.6-Cyber, added to its Daybreak initiative and scoped to authorized defensive work, is a vendor policy boundary. It is not a control that exists inside your environment.

Where not to let a model decide

The research reviewed here is deliberately unflattering to the marketing, and the disagreement is worth naming. Vendor framing says agents are ready for sustained autonomous operation. A survey of roughly 336 papers finds GUI and computer-use agents still lack error recovery, safety checks and auditability for sustained business use. Separately, hidden-activation probes for code correctness shift with the extraction method even when the bug is isolated, which means the probe is partly measuring the pipeline rather than the code. Neither belongs in a merge gate.

Keep execution-based verification as the source of truth. Keep the named human reviewer of record on every AI-assisted patch. That is the single control the kernel retained, and the one regulated buyers will start asking you to evidence. Budget explicit refactor capacity as well, because agents solve each task correctly and plan for nothing. That produces a debt curve rather than a one-time cost.

A model can tell you why the crash happens. It cannot be the reason the patch merged.