The Scientist

GPT-Red is a self-play RL red-teamer, and the detail worth more than the headline 84% is that its successful attacks fed straight back into training, hardening GPT-5.6 Sol without measured capability loss. That closes a loop manual review can't replicate: generate, exploit, retrain, at machine speed. A human team clearing 13% of the same scenarios isn't really a comparable baseline. It's a signal that quarterly pen-testing on this class of model is running on borrowed time.

Treat the number as directional, not precise. There's no disclosed sample size, scenario taxonomy, or attack-class breakdown, and the 84% figure is measured against GPT-Red's own scenario distribution, not against any external defense set. What it establishes is a ~6x detection gap between automated and manual red-teaming, which is a real signal for where eval budget should move, even before the methodology gets tightened up.

The regulatory pattern tracks the technical one, though it's worth being precise about what's correlation and what's cause. GPT-5.6 shipped under a staggered release the administration requested on security grounds; Anthropic drew the same scrutiny first, in June 2026, after a competitor flagged model vulnerabilities to officials. Model-level security review now functions as a release gate, which means frontier availability carries regulatory latency that isn't controlled by the lab shipping the model. In parallel, the agentic OSS ecosystem logged 30+ CVEs in eight weeks, while only about 21% of firms report mature agent governance. That gap predates this release cycle.

Sources agree on direction and diverge on rigor. The eval evidence is quantified; the regulatory reporting is single-source, with no technical detail on the vulnerability class involved. The thing this doesn't tell you is which failure mode triggered the staggered release, and that's the detail that would actually change a threat model. For evaluation harnesses generally: a report that covers accuracy and latency but skips attack-success-rate isn't measuring the dimension that enterprise buyers and regulators are both going to ask about next.

The mechanism is what's new. Effort is now installed by effort-conditioned SFT and mode-conditioned RLVR, per Raschka's read across six open-weight models. DeepSeek V4 trains Non-think / Think-High / Think-Max as separate modes with their own context windows and length penalties, then distills them into one checkpoint. Effort here is a learned behavior, not a decode-time knob, which means the flag only does something if post-training actually honored it. That's a regression test worth running before you trust the flag, and most teams skip it.

The part worth double-checking is the cost curve. On DeepSeek V4 the token cost inverts: higher effort spends fewer tokens, not more, because the model was trained to reason more economically at the top of the ladder. Kimi K3 compounds this from the other direction, emitting 21% fewer output tokens than K2.6 on equivalent tasks. Output-tokens-per-completed-task is a real cost variable now, and it moves independently of the per-token price on the rate card. That independence is the thing to check first if a forecast starts drifting.

The consequence: any $/M-token spreadsheet that treats a model as a single price point is stale. Cost-per-completed-task is a function of effort level, verbosity, and your task mix — three things the vendor's pricing page doesn't capture. Neither release shipped ablations isolating effort's contribution, so the size of that effect on your workload is still an open question, not a given.

Both sources agree output-token efficiency is the lever. That agreement is correlational across two model families, not a controlled result, and it doesn't substitute for validating the curve on your own suite before you rebuild forecasts on it.

Netflix's serving decision is the transferable data point here. The team moved TensorRT-LLM → vLLM in summer 2025, citing custom architectures, custom decoding hooks, and easier debugging. They then folded LLM inference into the same Model Scoring Service that already runs their XGBoost, TF, and PyTorch models. Triton still handles batching and GPU scheduling behind one control plane. The lesson isn't "adopt vLLM." It's that treating LLM inference as a silo apart from classic ML is the anti-pattern a mature shop dismantled in summer 2025.

Kimi K3 is the stress test for that flexibility. Its Kimi Delta Attention breaks conventional prefix caching, which is why Moonshot had to contribute a bespoke vLLM implementation, landing July 27, and why they recommend 64+ accelerator supernodes. A cost model built on the assumption that cached system prompts amortize across requests will quietly lose the pricing win here. That's not a benchmark you can borrow from someone else's paper. It's a cache-miss-overhead number you measure on your own prompt distribution.

A second trap sits in the memory math. Extreme MoE sparsity means K3 activates 16 of 896 experts per token, which cuts compute-per-token, not memory. Self-hosting keeps all 896 experts resident, so VRAM planning scales with the total 2.8T params while throughput scales with active params. The compute break-even against the $3/$15-per-M-token API is worth running before committing infra.

The operational substrate matters as much as the kernel. Once agents, not humans, are the ones reading a feature-store schema or model registry, stale metadata stops being a productivity tax and becomes a silent production incident. Serving discipline now has to span the control plane and the metadata it reads from.