The Scientist

The Signal

The number worth planning around is not the 5T headline. It is the lopsided per-language growth. C++ grew 15x over v2, TypeScript 7.5x, Rust 7x. Python only grew 4.8x, because it was already well-covered. A codebase leaning on systems, infra, or web code was starved of exactly those tokens in v2. That asymmetry is what tells you where a continue-pretrain or fine-tune spike actually moves the needle, rather than where the token budget happened to pile up.

Under the Microscope

Two operational changes matter more than raw scale. First, contents ship inline instead of Software Heritage IDs, which ends the blob-resolution step that cost a day of preprocessing in v2. Second, the dataset filters restrictively-licensed code — the same class of exposure AINews ties to Anthropic's reported $1.5B copyright settlement, the largest in history. A fresh GitHub recrawl through August 2025, a ready-to-train split, and a full bucket for your own dedup round it out. The license filtering is the cheapest insurance against training-data liability, and it tidies the reproducibility story too.

On the training side, vLLM's prime-rl 0.6.0 is a reference architecture for RL-on-agents at scale: FP8, expert parallelism, prefill/decode disaggregation, KV offload, routing. AINews reports it trains GLM-5 on SWE tasks at 131k context with sub-5-minute steps on 28 H200 nodes. Those are vendor-reported throughput numbers, not independently verified. The thing those step times don't tell you is what they replicate on your hardware. The config patterns — disaggregation plus KV offload as the levers for long-context rollout throughput — are the durable takeaway either way.

The third thread is quieter and points at a real convergence. PRO-LONG treats agent memory as a queryable database of full structured interaction history, and reportedly beats bespoke long-horizon memory harnesses on ARC-AGI-3 with fewer tokens. ARC-AGI-3 is a narrow slice, so the Pareto claim of higher completion at fewer tokens is a hypothesis to falsify, not a result to bank. Paired with the D1 dispatcher and the Harness Handbook, the direction is clear: from context-stuffing toward structured, retrievable memory.

In Your Pipeline

For code-model teams, The Stack v3 is the new floor. Prioritize fine-tune experiments on the C++/TypeScript/Rust gains where v2 was thinnest. That is where the marginal token buys the most. For long-horizon agents, PRO-LONG is a cheap spike with a genuine Pareto claim, worth testing against your own memory harness before committing to a rewrite. What none of these numbers tell you is how much holds on your task distribution. A one-sprint eval settles that.

The Stack v3 and vLLM prime-rl are the two items you can act on — everything else is a benchmark claim waiting for your own harness to confirm.

The Signal

Here is the mechanism most cost post-mortems miss. Most production stacks run one main LLM plus several small specialized models: an embedder, a reranker, an extractor. Standard serving (vLLM, TEI) pre-allocates GPU memory per process at startup with zero cross-process visibility, so a 600M-param embedder claims a full GPU it will never fill. That is why a 'move to smaller, better models' initiative can fail to register on the infra bill at all. It is a GPU-sharing architecture gap, not a model-quality tradeoff. Superlinked's open-source SIE (github.com/superlinked/sie) claims to fix it with shared memory pools and on-demand loading behind one API. Worth a one-day POC against your current stack. The footnote: the claim is vendor-sourced and unbenchmarked here.

Under the Microscope

The reason the last hardware upgrade disappointed is the roofline. An A100 delivers ~300 TFLOP/s FP16 but moves only ~2TB/s between HBM and compute, and autoregressive decoding reads the full model weights plus the entire KV cache from memory for every token. Swap in an H100 with 3x the FP16 compute and token generation barely moves. This is a memory-bandwidth problem dressed as a compute-upgrade decision. Two serving techniques attack that directly:

• Paged attention cuts KV-cache memory waste from 60-80% to under 4%, per the original vLLM paper. A stack not already running it is likely leaving 60%+ of KV-cache capacity on the table.
• Speculative decoding is a throughput win with no quality tax. The rejection-sampling scheme is provably distribution-identical to the target model alone, not an approximation.

Caveat: none of these figures are re-benchmarked; the 60-80%→4% number is a paper citation and the others arrive with no fresh throughput data. Treat this as a mental-model refresh, not new evidence.

On the demand side, TLDR Fintech reports Ramp's customer base grew AI token spend 20.7x since June 2025, with early users of its spend-control product finding up to $10K/month from prompt caching and one in three identifying cheaper model swaps for the same task. The $10K figure has no disclosed denominator and the downgrade recommendations ship with no quality-preservation methodology. The thing these numbers don't tell you is where they came from. What they do tell you: enterprise LLM cost management is now its own product category, scaling faster than most teams' observability.

In Your Pipeline

Two experiments fall out of this at near-zero cost. First, run a roofline/arithmetic-intensity profile on the actual workload before any GPU spend, and confirm paged-attention serving is on. That is a measurable, fixable throughput ceiling. Second, instrument prompt-cache hit rate on the highest-volume repeated-context endpoints (RAG retrieval prompts, system prompts, few-shot templates) for a week and measure the token-cost delta. Then build a lightweight model-routing eval: compare output quality at the current tier and one tier down on a sample of production tasks, validated against your own quality bar rather than a vendor's black-box recommendation.

Buy bandwidth, not FLOPs — and don't expect 'switch to smaller models' to cut costs until your serving stack can actually share a GPU across them.

The Signal

Every compromise this cycle beat the layer teams assumed was the backstop. That is the reason to treat the cluster as a same-week response and not a backlog ticket.

Under the Microscope

RefluXFS is the priority. It is a race condition in XFS reflink handling that lets an unprivileged local user overwrite any readable file for passwordless root. It is present in every kernel since v4.11 (2017) with reflink=1, estimated across ~16.4M systems — default RHEL, Oracle Linux, Amazon Linux, and Fedora. TLDR InfoSec, Risky.Biz, and Cyberpresso corroborated it independently. It is reboot-persistent and writes no kernel log, so the SIEM has nothing to alert on. The thing that number doesn't tell you is which of those systems you actually run; Amazon Linux 2/2023 nodes backing EKS and SageMaker are named as default-affected. For multi-tenant GPU clusters that lean on user isolation, this breaks the threat model outright. The only mitigation is confirming kernel version and rebooting, before the next long-running training job, not after.

LiteLLM, a package many teams sit in their multi-provider routing or observability stack, was compromised in March 2026 through a mutable reference in the Trivy GitHub Action. That is not a code vulnerability in LiteLLM itself. An attacker altered what a pinned-by-tag Action actually executed. PyPI's structural answer, blocking new file uploads to any release older than 14 days, is low-friction: Devshot notes only 56 of the top 15,000 packages needed a post-window wheel. Sandworm_Mode (CrowdStrike, Feb 2026) fills in the rest. It is purpose-built to blend into high-frequency agent traffic, inserts multi-day delays before follow-on activity, and self-destructs when it can't propagate. A quiet first week tells you almost nothing.

Underneath all of this sits the provenance liability Risky.Biz flags. The Anthropic settlement detailed in the Stack v3 dive sets a pricing floor on unlicensed training data. SectopRAT being served from Anthropic's own Claude domain retires domain reputation as a trust signal.

In Your Pipeline

Three reports converging on 'audit the infra layer, not the model weights' is the actionable pattern. The kernel fleet, the package registry, and the CI toolchain are the path attackers actually take. None of them shows up in a model eval.

A frontier model's own test sandbox failed exactly like a container's kernel isolation failed: the control everyone assumed was authoritative wasn't.

The Signal

OpenRouter's last round put it at $1.3B. A ~$10B bid is a 7.7x markup, and Databricks was in early discussions too. Nobody pays that for a thin abstraction over provider APIs unless they see downstream revenue the current price doesn't capture. Structurally, that points at pricing pressure on the exact layer many teams lean on for failover and cost arbitrage. The thing the markup doesn't tell you is who wins, and the acquirer's identity decides what breaks for you:

Under the Microscope

The multiple says nothing about timeline, so price the defensive move on its own. OpenRouter is a thin abstraction over provider APIs, which means a provider-agnostic fallback (LiteLLM or direct provider SDKs) is a days-not-weeks migration if you build it before you're forced to. The success metric is measurable: reroute 100% of LLM calls off it inside one sprint. Second-order irony worth logging — LiteLLM, the obvious fallback, is the same package compromised via a GitHub Action, so the fallback path still inherits the CI hygiene from the supply-chain thread.

Capacity rhymes with this. AMD locked in Anthropic as an anchor customer — up to 2GW of Instinct MI450 starting H1 2027, backed by up to $5B in milestone-based AMD investment — the first credible non-Nvidia inference commitment at this scale, per TLDR. It changes nothing in a stack running today. It changes multi-year cost models and lock-in analysis. Eighteen months of lead time is enough to find the ROCm-vs-CUDA tooling gaps before capacity ships rather than after. The frontier field is consolidating in parallel: The Information notes Amazon closed its San Francisco AGI Lab and Sierra acquired agent startup Takeoff, so value is migrating up into the routing and agent layers where applications live.

Caveat: the OpenRouter deal is reported as talks, not closed, and the strategic-motive read is analytical inference, not disclosed roadmap.

In Your Pipeline

Provider portability is now a first-class engineering requirement, not neutral plumbing. An acquisition is a repricing and terms-of-service event on the acquirer's clock, not yours. Keeping at least one open-weight model as a first-class citizen in the eval harness is what stops vendor concentration from cornering you when the closed-model choice set narrows.

When a payments company pays $10B for a model router, the layer between you and your LLMs stops being neutral plumbing.