The Scientist

What Shipped

Multi-token prediction moved from papers to serving stacks inside a week. Gemma 4 ships with trained-in draft heads. llama.cpp merged MTP support (PR #22673) for Qwen3.x. vLLM, SGLang, MLX, and Ollama all have day-0 or beta support. The drafter is small enough to be noise in the memory budget: 78M parameters against a 27B+ target.

The Numbers That Matter

Community benchmarks on llama.cpp's beta with Qwen3.x report a ~75% acceptance rate at 3 draft tokens and >2× token-generation throughput. Google claims up to 3× on Gemma 4 with zero quality degradation. The numbers are real. They are also batch-size-one, single-stream numbers, which is not the regime most of us run in.

On a loaded production server the sequential-step bottleneck shrinks, GPUs are closer to saturation, and the 2-3× headline compresses to 1.3-1.5×. That is still worth the migration. It is not 2-3×.

Where It Breaks

Acceptance rate is distribution-sensitive. MTP does well on repetitive, predictable token sequences: natural language, common code patterns, JSON. It degrades on:

• Code with long unique identifiers
• Structured output with rare keys
• Languages the draft head was undertrained on
• High-concurrency serving where the GPU is already saturated

The failure mode is paying the draft-model cost for zero speedup on mismatched distributions. The fix is routing. Send MTP-friendly requests through the speculative path and keep standard decode for the rest.

Cross-Source Agreement

Five independent sources converge on the same read: MTP is real, production-viable, and worth a spike this week. They converge on the same caveat too. Measure acceptance rate per request type, not aggregate tokens per second. Aggregate throughput hides the slices where the drafter helps nothing, which is exactly the slice you need to price.

Why This Week, Not Next Quarter

Most inference optimizations come with a tax: retrain, requantize, or eat a quality delta. MTP, when acceptance is high, requires no retraining, no quantization trade-off, and no quality degradation. The Gemma 4 E2B drafter is 78M parameters, smaller than most embedding models. Integration is hours on the supported frameworks. A few hours of engineering for a possible 30-50% cost reduction on qualifying traffic is an asymmetry worth taking, assuming your traffic qualifies. Measure before you assume it does.

The Convergence

The shared root cause across this week's four agent-security incidents is simple: AI agents have filesystem, network, and tool access but operate under trust assumptions designed for humans. The threat model shifted along two axes at once: what agents do when hallucinating, and what attackers do once agents hold permissions.

The Attack Surface, Decomposed

Why This Matters for Data Scientists Specifically

Hallucination rate was a quality metric last quarter. This week it became a security metric. Every percentage point of fabricated imports in a coding agent's output is a percentage point of pre-positioned attack surface. The economics favor the attacker. Registration is free, detection requires defenders to notice a package they never intended to depend on, and the agent loop was designed to remove humans from exactly that decision.

A coding agent's hallucination rate is now a security metric. Teams with dependency allowlists and trace review can bound their exposure operationally. Teams without either have no allowlist blocking a fabricated import at resolve time and no trace review catching it post-hoc.

The MCP Problem Is Architectural

OX Security's finding — 150M+ downloads, 10+ CVEs tracing to one root cause — is not a bug in the usual sense. STDIO transport runs subprocesses on the host and pipes them stdin/stdout. A config file pointing at a hostile binary gets executed with user privileges. No sandbox sits in the path by default. The only production-ready containment pattern is Sondera's Cedar-policy-on-Claude-Code-hooks, which pushes enforcement to the action layer rather than the prompt layer.

What's Different From Last Week's Supply-Chain Coverage

Monday's PyTorch Lightning compromise was a traditional supply-chain attack: legitimate package, compromised binary, 42-minute window. Slopsquatting is structurally different because the attack surface is model-specific and prompt-determined. A hallucinated package name that Copilot emits at 0.2% frequency becomes a deterministic target the moment an attacker enumerates it. The distribution is larger than typosquat targets and harder to bound.

The Capability-Reliability Gap Widened Again

This week's evidence converges on a gap between capability scores and production reliability that has been widening for eighteen months.

1. GPT-5.5 Instant jumps +15.8pt on AIME 2025 (65.4→81.2) and +6.8pt on MMMU-Pro. The capability gains are real.
2. Opus-4.5 with web search still produces ~30% ungrounded claims in multi-turn conversations, meaning the strongest model with the best retrieval hallucinates roughly one turn in three.
3. Kapoor/Rabanser/Narayanan (Feb 2026) benchmarked 14 frontier models over 18 months and found capability rose sharply while reliability improved only modestly or not at all.

Capability continues to scale while reliability does not, which means every metric in the eval harness that conflates the two is now actively misleading.

Why This Gap Persists

The mechanisms are distinct. Capability gains come from better reasoning chains, more training compute, and richer RL signals. Reliability requires calibration, which is a different training objective that most RLHF pipelines don't explicitly optimize for. Google's "faithful uncertainty" work reframes hallucinations as failures of uncertainty expression rather than knowledge gaps. That is the correct diagnosis.

The Multi-Turn Compounding Problem

A 30% single-turn ungrounded rate compounds badly. Each turn's ungrounded claim contaminates the next turn's context, so by turn 5 in a typical RAG pipeline, effective accuracy is materially worse than the turn-1 dashboards suggest. Teams measuring only per-response hallucination rates are missing the compounding effect entirely.

The Practical Consequence

OpenAI's claimed 37.3% hallucination reduction on GPT-5.5 is measured against user-flagged errors, a biased and non-reproducible signal that overweights surface-level mistakes. Silent hallucinations, fabricated citations, and subtle domain errors are systematically underrepresented. The vendor's number answers a different question than the one a production system asks.

Sources disagree on whether GPT-5.5 is a meaningful upgrade for grounded QA workloads. The AIME gains are unambiguous for math and reasoning. The grounding improvement is claimed but not independently measured. My read: GPT-5.5 is a net upgrade for reasoning-heavy workloads and roughly a wash for factual grounding, where retrieval augmentation is still doing most of the work.

What Google's "Faithful Uncertainty" Reframing Changes

Scoring accuracy without scoring calibration measures the wrong bottleneck. A model that is wrong 10% of the time and knows it is routable to a human reviewer, which is a different production object than one that is wrong 10% of the time and confident about it. The eval harness should reflect that distinction before the next model refresh, not after.