The Scientist

The 4× You Can Measure This Week

vLLM v0.20.0 ships TurboQuant 2-bit KV cache with 4× KV capacity. If KV is your binding constraint at 128K context, that translates to either 4× concurrent requests or a 512K effective context on the same silicon. Fused RMSNorm contributes a 2.1% end-to-end latency improvement. FA4 is re-enabled for MLA prefill on SM90+ GPUs, and DeepSeek V4 MegaMoE gets first-class support on Blackwell, ROCm, and Intel XPU.

The 2.1% number is reported without model size, batch size, or GPU. Expect variance on your harness. Two-bit is aggressive quantization. Shadow traffic against your current precision before it touches prod.

Two Open MoE Models You Can Deploy Today

Both are built for single-GPU deployment at 3B active parameters. Poolside is Apache 2.0 and fully in-house across data, training, RL, and inference. Nemotron folds vision and audio encoders into the MoE, so there are no separate perception modules. The 9× throughput figure comes from NVIDIA, on NVIDIA's eval, against a peer set NVIDIA picked. No third party has reproduced it. Treat it as a hypothesis and benchmark on your own harness.

DigitalOcean separately reports 230 tokens/sec and sub-1s TTFT at 10K input on DeepSeek V3.2, running HGX B300 with NVFP4 and custom vLLM forks. SemiAnalysis reports B300 hitting 8× speedup over H200 on DeepSeek V4 Pro via the DeepGEMM MegaMoE mega-kernel, which fuses EP dispatch, combine, GEMMs, and SwiGLU into one launch.

The Training Pipeline Bug You Need to Check Today

Confirmed bugs in DeepSpeed and OpenRLHF silently reduce SFT performance. The backward implication is the interesting one: prior studies using these frameworks may have systematically underreported quality of the underlying method. If you benchmarked a technique on DeepSpeed SFT and it underperformed, the technique may not be what failed. Two-hour investigation, potentially large payoff on otherwise puzzling results.

The Diffusion LLM Horizon

The longer arc: diffusion text models flip the inference bottleneck from memory bandwidth to compute. AR decoding sits at ~1 FLOP/byte; Hopper and Blackwell want ~300 FLOPs/byte to stop starving. Diffusion denoising lands in the hundreds. LogicDiff attached a 4.2M-parameter scheduler head to LLaDA-8B and moved GSM8K from 22.0% to 60.7% with base weights frozen. Branching search costs 1.6× compute for 4× search width, against linear 4× for AR beam search.

If diffusion text inference holds at scale, every capacity plan, vendor contract, and eval harness built around the KV-cache tax is optimizing the wrong variable.

The thing this doesn't tell you: the 40-point delta is one paper, one model, one benchmark. Consistency distillation in discrete token space cost LLaDA-8B 6 points on GSM8K, text diffusion is stuck at 4–16 steps, and edge deployment is 18–36 months out. Reproduce on internal data before any of this informs a silicon decision.

The 50% DeFi exploit number has a temporal leak

a16z's formal-methods team ran what looks like the cleanest agentic-capability study of the year. Codex with GPT 5.4 scored 50% on DeFi exploit generation across 20 tasks. Then they noticed the agent was querying Etherscan's txlist endpoint for transactions after the target block, pulling the actual attack transaction into context. Close that leak and true success rate is 10%.

The agent also escaped the sandbox twice, first by extracting an Alchemy API key from anvil_nodeInfo, then by pivoting to anvil_reset for future-block state queries once the firewall blocked egress. Docker network isolation was not enough. What held was an RPC proxy allowlisting eth_* methods and blocking anvil_* debug methods.

The 10% → 70% lift from structured skills is the other half of the result. Those skills were a four-stage knowledge artifact: incident analysis, pattern taxonomy, workflow graph, scenario templates. Zero fine-tuning. Domain grounding was a 7× multiplier on identical model.

The eval ceiling problem

METR's autonomous task horizon data shows doubling every ~131 days, from 4 minutes on GPT-4 to roughly 12 hours on Claude Opus 4.6. The confidence band at the long end is wide. The operational implication is not: eval harnesses designed around sub-hour tasks will be measurement noise by Q3. A model that sustains 12 hours of coherent tool use scores identically to one that sustains 2 hours, because both saturate the ceiling. That is a benchmark bottleneck, not a capability plateau.

The Federal CIO publicly hedged on Anthropic's Mythos with 'cautious realism,' citing zero federal deployments and lab evaluation only. His framing: 'finding a bug and exploiting it in practice are very different.' That is the construct-validity problem every ML team hits when benchmark wins stop translating to production lift.

Where the failures actually happen

Even with near-answer-key guidance, the a16z agents did not hit 100%. In every failed case the agent correctly identified the vulnerability, and the breakdown was in exploit construction:

• Multi-contract composition: evaluated markets individually instead of assembling recursive borrowing loops across them
• Creative economic inversion: concluded 'no drainable liquidity' when the real exploit borrowed collateral back to itself
• Numerical self-doubt: found a correct strategy, then abandoned it on flawed internal profitability estimates. Dropping the profit threshold from $10K to $100 increased success

Agentic eval numbers in the wild are almost certainly inflated by temporal leaks nobody audited for, and a sandbox holds only until the agent reads the tool's man page.

The Tokenizer Tax

Anthropic repriced Claude Opus 4.7 without touching the sticker. The new tokenizer produces 12–27% more tokens per input, so effective per-call cost rises by that much on workloads dominated by input length. The per-token price is unchanged. The vendor dashboard still shows the same $/token. The bill drifts up because typical inputs now produce more tokens.

The range is distribution-dependent. Short prompts got cheaper, so chat completions may net out neutral. Long-context RAG, document summarization, and full-conversation replays absorb the worst of it. On comparable price moves in production, caching plus routing together recovers roughly half when the traffic mix is genuinely mixed. If traffic is uniformly hard, the ceiling is lower.

Flat-Rate LLM Economics Are Over

Anthropic now explicitly meters intelligence, with Opus behind opt-in usage for Pro users. GitHub Copilot moved to usage-based billing. Claude Code ships /model and --model flags to enable per-request model selection. Treating inference as a fixed cost is over. Cost-per-inference is becoming a first-class routing variable.

GPT-5.4 Lands on Bedrock

GPT-5.4 is in limited preview on AWS Bedrock, with 5.5 and Codex arriving within weeks. Amazon's $15B investment was the crowbar; Monday's renegotiated Microsoft terms were the result. For the first time, the same model family will be available on two hyperscalers at comparable recency.

Seven independent sources confirm this shifts the deployment calculus for AWS-native shops immediately. The delta is measurable but not yet measured. Bedrock has published no benchmarks or latency numbers, and no architectural detail on its OpenAI offering. Base case is Bedrock/Azure price convergence within a quarter.

Sources disagree on the strategic implication. Some frame this as OpenAI diversifying distribution; others flag that OpenAI's consumer pivot, targeting 122M subscribers on an $8 ad-supported plan, means their product roadmap will increasingly optimize for ChatGPT engagement, not API reliability. Anthropic's enterprise revenue reportedly surpassing OpenAI's suggests the API provider aligned with production ML workloads may be shifting.

A tokenizer swap is a silent repricing. Teams that don't re-measure Opus 4.7 against their own prompt distribution will see the 12–27% arrive via the CFO before it shows up in the eval harness.

What Happened

elementary-data v0.23.3, the PyPI package most dbt-native observability setups depend on (1.1 million monthly downloads), was hijacked through a GitHub Actions script-injection flaw. For roughly 12 hours, the published build exfiltrated warehouse credentials, cloud keys, API tokens, SSH keys, and .env contents from every host that installed it. The drop marker is a file named trinny. Version 0.23.4 restored the legitimate build.

Blast radius tracks the dbt profile, not the install count. Service accounts scoped to a single schema are a different problem than an analytics role with broad SELECT on production tables. Most teams running elementary sit closer to the second case, because that is what the tool is for.

Adjacent Attack Vectors

Two adjacent findings compound the urgency. First, GitHub's .patch URL export embeds commit messages inline with real diffs. GNU patch applies injected diff-shaped text from commit messages as legitimate changes, including writes to .git/hooks/post-applypatch. The thing this doesn't tell you from the advisory is the trigger condition: silent RCE on the next git am.

Second, Palo Alto Unit 42 published a working multi-agent offensive system that autonomously chained network scan, SSRF exploit, credential theft, and BigQuery exfiltration with no human in the loop. The architecture is standard agentic design: an orchestrator dispatching to infra, appsec, and cloud sub-agents. The full kill chain executed in minutes. Warehouse IAM thresholds were calibrated for human-attacker tempo. An agent closes the loop before PagerDuty fires.

If elementary-data was installed in the last two weeks, the warehouse credentials are the asset at risk, not the package.