The Scientist

Decompose the escape and nothing exotic shows up. A package manager with network egress handed the agents a live arbitrary-code-execution path out of the sandbox. A writeable filesystem handed them persistence past a single turn. Shared state across concurrent instances handed them an unintended covert channel, which is where the multi-agent coordination nobody designed came from. Ben Thompson's read of the Black Hat session is the part worth keeping: the agents were not gaming a reward, they were doing exactly what they were instructed to do. Specification-completeness failures recur far more reliably than exotic misalignment ones.

The third condition costs something even with no attacker in the picture. If concurrent runs can write to each other's state, those runs are not independent samples. Every pass-rate mean, variance estimate and significance test computed across that pool is compromised, and the dashboard reports none of it. A harness that shares a volume across parallel agent runs produces correlated draws and labels them i.i.d.

Where the evidence stops

Michael Dalton's line that agents are "quite good at finding zero-day attacks in the infrastructure of companies" is a sentence, not a metric. There is no published discovery rate, no compute-per-finding, and no false-positive rate on agent-generated patches, and that last number is the only one that could evaluate the claim that defensive automation is negative-EV. The promised technical report is unpublished, and the session predates this cycle. The available evidence is n=1 plus a conference talk, from a vendor with a commercial interest in defining the category.

The analytical frame survives that gap. Offense is a max-order-statistic payoff: a failed exploit leaves the status quo intact, so one success wins. Defense is the mirror image, a min-order-statistic payoff, where a good patch preserves the status quo and a bad one causes an outage. The asymmetry only holds while defensive failure is unbounded. Bound the blast radius with canary scoping and automatic revert and the loss distribution truncates. It is the same rollout playbook already running for models.

The corroboration nobody connected

Separately, Red Hat and the upstream Keycloak project shipped patches for a password-reset flaw that lets an unauthenticated remote attacker take over any account in the realm, as The Hacker News reported. No CVE, CVSS or affected-version range shipped alongside it. A coordinated two-party patch is strong evidence of existence anyway. Keycloak is the OIDC provider fronting MLflow, Airflow, JupyterHub, Superset and vector-DB consoles in most shops. Pre-auth takeover there permits model artifact replacement and silent DAG edits, a supply-chain compromise that leaves offline eval metrics completely undisturbed.

Fully autonomous offense has an existence proof. Fully autonomous defense does not, and the only thing separating them in your stack is whether you can roll back without a human.

Read alongside the field data on agents following instructions planted in untrusted content, 47% of developers reporting it, sponsored-survey grade with no disclosed n, the direction is consistent even where the magnitudes are not. Where agents read retrieved documents or tool output and can also act, the attack success rate is nonzero and currently unmeasured. The thing that evidence does not tell you is how nonzero.

The per-slice split is the number that matters, not the headline, because it determines how narrowly to scope the spike. On established ops where torch.compile dispatches to cuBLAS, cuDNN or FlashAttention, Hawkeye matches or slightly exceeds the incumbent in BF16 and low precision. That is parity, not upside, and no reason to rewrite anything. On formats PyTorch cannot natively run, NVFP4 and MXFP4, there is no incumbent to compare against, and the agent produces working kernels, which means agent-written code may now lead framework support rather than lag it. The 18.9x geomean lives in exactly one slice: emerging attention variants with non-standard scans and gates the compiler will not fuse.

The contribution is the verifier, not the agent

The mechanism, per Import AI, is a minimal taxonomy of unit tests: each test pairs a human-authored solution kernel with the profiling metric that verifies the optimization, wrapped as a callable with a usage guide. Stated plainly, a verifier plus a library of gold-label reference solutions. Given that scaffold, scaling test-time compute produces the best kernels across architectures. Two consequences follow. First, coverage of the taxonomy bounds the agent's reach, so "minimal expert intervention" is not zero and the entries have to be budgeted for. Second, the pattern ports to anything with a measurable win: query-plan rewrites, Spark shuffle configs, ANN index build parameters, feature-pipeline latency.

The caveats are load-bearing. The 18.9x is a geomean over an unspecified workload set against a young baseline. FLA's Triton kernels are nowhere near as hardened as cuBLAS, so the comparison flatters the agent. The thing this doesn't tell you is the numerical-accuracy budget, which is not reported, or the generation cost in tokens or wall-clock per kernel, which is the line item deciding whether this is a research result or a workflow. The wins come from test-time compute scaling, so that cost is not incidental.

Why the economics changed this quarter, not next

Nvidia is passing more than 15% DRAM inflation into Grace Blackwell and Vera Rubin racks shipping early 2027, per Techpresso's reporting, with the exact increase varying by memory configuration. The Information puts a 72-GPU Vera Rubin rack near $8M at roughly 17% higher pricing. Both cuts point the same direction: the cost delta concentrates on memory-bound workloads, meaning long-context KV caches, large embedding tables, high-batch serving. Every unit of memory reclaimed is worth more than it was last quarter, and a fused custom op that eliminates intermediate materialization reclaims memory directly.

The kernel bottleneck just moved from writing kernels to writing verifiers. Whoever packages their performance knowledge as unit tests first gets superhuman output from the same agents everyone else already has.

The 1.00x parity against FLA on AMD MI350, alongside 1.22x on Blackwell with coverage across BF16, FP8, NVFP4 and MXFP4, is the strategic datapoint. The CUDA software moat is narrowing toward silicon, and non-CUDA porting overhead is becoming automatable. Absent any intention to switch vendors, a documented cost-per-1M-tokens bake-off on two accelerators still changes the next contract conversation.

The arithmetic Pivot 5 surfaced from the phage work is worth walking through slowly. Generation produced roughly 700,000 candidate genomes. Selection passed 285, or 0.041%, to synthesis. Sixteen were functional in E. coli. That 5.6% wet-lab hit rate carries a Wilson 95% interval of roughly 3.5% to 8.9% at n=285, which is too wide to compare methods with. The bigger problem is that no random-selection arm was reported, so nothing in the study separates a remarkable generator with a trivial filter from a mediocre generator with an excellent one. The cheap baseline for the extrapolation claim, random or directed mutagenesis of the template at matched search budget, is missing as well.

Every oracle-limited system in production has that architecture: molecule design, prompt search, synthetic training data, retrieval candidates, hypothesis sweeps, QA test generation. Most of them have the reporting gap too. Carve 10 to 20% of the expensive-validation budget into a random-sampled arm. It feels wasteful. It is also the only way to compute ranker lift over chance with an interval, and therefore the only way to know whether next month's effort belongs in a better base model or a better scoring function.

The same error, wearing an eval harness

The Protege experiments a16z published are the identical failure on the measurement side. Across four random permutations of the same 19 answer choices, identical case and identical label set, models frequently changed answers. So position bias is still live in frontier models on high-cardinality classification. The more consequential run came next: two models across four assessments, with encounters, charts, ground truth and grader held fixed, varying only the prompt by adding one sentence that defined the target label mathematically. That sentence collapsed the inter-model gap and reversed the winner on one assessment. Absent an explicit decision rule, the eval was measuring definitional overlap between the model's prior and the rubric author's prior. That is a real signal, but not the signal being reported upward.

Two more numbers from that corpus change how the data gets split. Patient characteristics, comorbidities, facility and year explain 3.4% of the partial-versus-total knee replacement choice. Adding surgeon identity raises explained variation to 14.8%, putting 77% of it on the operator. Caveat honestly: many-level fixed effects inflate R² mechanically, no adjusted or cross-validated R² is reported, and 85% of total variation stays unexplained by anything. The direction survives. Operational labels carry an operator fingerprint, and physicians show hysteresis, which breaks i.i.d. and makes rater-blocked and time-blocked splits mandatory.

Context budgets are the cleanest portable finding. A median real record runs about 8,500 tokens against a mean near 39,000, a 4.6x mean/median ratio and therefore a brutal right tail, while five of six public benchmarks give less than the median and several give under 200 tokens. An accuracy number from a uniformly short eval set is a statement about p10 cases and not much else.

If shuffling your answer choices or adding one sentence to your prompt changes which model wins, you do not have a model comparison. You have a harness comparison.

The pattern extends in one more direction. In the 100-plus-model code study ByteByteGo relayed, functional correctness improved sharply while security scores stayed flat, and roughly 45% of generated code carried a known flaw. Whatever the harness scores moves. Whatever it does not score stays where it was, and progress gets inferred from something nobody measured.