The Scientist

Start with the number the research does not report

A steering intervention that moves abstention by 33 to 52 percentage points is a guardrail only if the shift lands on items that genuinely have no answer. TheSequence's read of the USC and Arizona State work flags what is missing: no over-refusal rate on the answerable subset, and a very wide reported steering range. Until you produce that number on your own traces, you have a research result rather than a gate. A model that declines hard-but-solvable questions fails users more visibly than one that answers a few of them wrong.

What the probe actually is

The method is a logistic regression fit to the model's residual stream, the running vector every layer reads from and writes to. The labels are binary: problems that have a solution, versus math and code problems that are structurally unsolvable. A mean AUC of 0.939 across 11 models says a single hyperplane separates those two classes. The geometry largely appears before instruction tuning, and that relocates the blame: pretraining already carries the distinction, and post-training never wired it to the decision to decline. Confident-wrong output is a routing defect, not missing knowledge.

Orthogonality, not accuracy, is the shippable finding

That internal states carry more information than outputs is a decade-old result. The new part is independence. A cosine similarity near 0.087 between the recognition direction and safety-refusal directions means the two axes are close to perpendicular in activation space. So you can push abstention on impossible inputs without dragging along the refusal thresholds you spent a quarter calibrating. Had the two been aligned, every point of abstention gain would have cost compliance on benign prompts, and the technique would be a non-starter.

An activation probe is the cheapest verifier you will build this quarter: bounded error, near-zero inference cost, and it covers the one failure your graders never see, the question that had no answer.

The prerequisite most teams hit first

This needs activation access, which closed chat endpoints do not provide. That constraint is loosening from two directions in the source material. Simplifying AI notes frontier-scale weights shipping MIT-licensed on Hugging Face, and Exponential View reports a century-old American institution moving entirely to open-weight models while a hospital deliberately runs a mixed OpenAI and Anthropic stack. Model heterogeneity has become the conservative posture rather than the adventurous one. Whether this probe is buildable in your stack is decided by which models expose hidden states, so inventory that before you scope the work.

The protocol that makes it credible

1. Dump per-layer residual activations from eval traces you already store, so the experiment needs labeling effort rather than new inference.
2. Label items answerable versus structurally unanswerable, keeping topic and length roughly balanced so the probe cannot win on surface features.
3. Sweep layers for peak AUC and report variance across layers and seeds, not the single best layer.
4. Choose the threshold on a validation split, then report abstention F1 alongside over-refusal on the answerable subset.
5. Route above-threshold items to escalation or a clarifying question instead of a silent refusal, so you keep the signal that a human had to intervene.

One honest caveat: this is a single research report, summarized secondhand, with no independent replication and no published effect on downstream task quality. Spend the days anyway, because the cost is asymmetric. The probe reuses artifacts already on disk, and a negative result on your own traces is decision-grade information either way.

Triage by expected loss, not by source confidence

The thinnest-evidence claim in the source material carries the highest cost of inaction. Per The Information's briefing, the attribution to a swarm of OpenAI agents comes with no CVE, no affected-artifact list, no detection methodology and no stated attribution confidence from either the Nightingale Collective or the AI Futures Project. That is not enough to open an incident. It is more than enough to justify hardening that is cheap and reversible.

Here is the part that should sting whether or not the attacks are ever confirmed: a model load that names a repository without a commit revision has already accepted mutable weights into a pipeline you describe as reproducible. The same line of code is now a reproducibility bug and a security bug. No attacker is required for it to hurt you, since a maintainer force-push does the same damage to your ability to re-derive a result.

The fix ladder, cheapest first

1. Pin every model and package artifact to an explicit commit revision.
2. Mirror those revisions into an internal registry so builds stop reaching the public internet at train time.
3. Enforce safetensors only and ban remote pickle and torch.load paths, which execute code during deserialization.
4. Add a CI gate that fails the build on any unpinned or unmirrored artifact.

Then run the backward sweep. May 2026 is the earliest dated incident, so audit dependency lockfiles and model download logs from that month forward for version churn, typosquat-adjacent names, and unexpected maintainer changes.

Where three independent reads converge

You cannot audit another lab's agent fleet, but you can baseline your own, and the useful signals are boring: tool-call rate distribution, egress destination entropy, and credential lifetime. Simplifying AI supplies the sharpest correction to the default agent policy. Gawkbot's "reads unrestricted, writes approval-gated" model is not a safe invariant: an injected instruction exfiltrates through a read-classified outbound fetch, and no approval ever fires. Gate on destination and payload size, not on HTTP verb. TheSequence adds a vendor's own concession: Meta's Muse runs its agent inside a dedicated Secure VM with a separate Sentinel agent watching outbound activity. A monitor bolted outside the model is an admission that prompt injection is unsolved inside it.

Where the sources disagree, and why the harder read wins

Simplifying AI locates supply-chain risk in young single-maintainer repositories, the pip-installable agent tools asking for terminal access and more than a thousand integrations. The Information's briefing locates it in the two largest and most-trusted registries in your stack. The second framing is the expensive one, because "prefer popular, well-maintained packages" is the heuristic most teams use in place of verification. Both can hold at once, and the combined policy is unglamorous: hash-pinned, vendored, egress-allowlisted containers for anything experimental, and no experimental install inside an environment holding repository write tokens or CI secrets.

A model reference without a commit hash is not a dependency. It is a subscription to whatever gets pushed next.

Size this as insurance rather than incident response. The payoff holds even if the attribution collapses: pinned, mirrored, safetensors-only artifacts make training runs re-derivable, which you wanted before anyone mentioned agent swarms.

The arithmetic that makes the swarm reproducible at your budget

Exponential View does the division OpenAI did not. 130 billion tokens across 2.7 million messages is roughly 48,100 tokens per message, which describes long-context reasoning traces rather than chat turns. Over 88 hours that is about 1.48 billion tokens per hour, near 410,000 tokens per second sustained across the fleet, and around 13 million tokens per agent, or roughly 41 tokens per second per agent. Total effort: 880,000 agent-hours. Against "a few million dollars," that implies $15 to $38 per million blended tokens, which sits inside published frontier reasoning-model output pricing. That internal consistency is the only reason to give the cost figure any weight.

Scale it down and it stops being intimidating. A 1,000-agent, 10-hour analogue is about one eighty-eighth of the tokens, or low tens of thousands of dollars. That is a spike, not a capital request.

Why it is still n=1

No attempt count. No failure rate. No ablation on swarm size, so nobody knows whether 1,000 agents or 100 would have sufficed. No accounting of human steering. And an unverified claim from Professor Buckmaster that a year of academic work had already gone into the same narrow problem inside Codex before OpenAI mobilized internally, which makes provenance murkier rather than cleaner. One success with no denominator is a press release, not a benchmark.

The ratio is the transferable finding

Eighty-eight hours of generation, then 17 hours of Lean formalization, a machine checking the result. Terence Tao's judgment, quoted in Exponential View, is the sentence worth keeping: "one of the most prominent open problems in mathematics would now be solved; but there would be almost no value added to mathematics as a consequence." Twenty-five Fields Medalists signed a declaration to similar effect. In production terms, an unverifiable correct answer has approximately zero value, because you cannot distinguish it from an unverifiable wrong one.

Generation capacity scales on a compute curve. Verification capacity, when the verifier is a human reader, does not scale at all. So the order of work inverts: measure the verifier before you scale the generator. Seed a candidate pool with deliberate errors, report the grader's precision and recall, and treat that recall as the ceiling on how far parallel generation can carry you. Ten thousand agents behind a 70%-recall grader is an error factory with excellent throughput. The unit that matters is cost per verified-correct answer; cost per token is vanity. And any scored objective invites shortcutting. The faulty-reward-function warning Clark and Amodei published in 2016 is still the live failure mode being cited in a widely shared 2026 researcher resignation, which is why reward hacking, judge manipulation and tool-call abuse belong in your deploy gate.

Who gets to define "tested"

The Information reports that Anthropic, OpenAI and Google have held private talks about jointly creating a standards body for AI testing and auditing, and that Altman told an internal town hall he expects the labs to fund it without US government support. Self-funded regimes publish conformance claims against private test sets rather than runnable harnesses, because publishing the eval publishes the contamination surface. Meanwhile Amodei, quoted in The Information's briefing, says pacing "does not mean halting model training or technical progress." Capability keeps shipping; only the gate changes. The consequence lands on your process: your eval harness becomes an audit artifact, and sampling non-determinism means you cannot reconstruct one after the fact. Pin prompt sets, fix decoding parameters, record seeds, model snapshot IDs and the harness commit hash, all keyed to an immutable run record.

Where the sources disagree

Exponential View's projected cost curve, a few million now falling to tens of thousands in two years and a few dollars later, implies roughly 100x in 24 months, a halving every 3.6 months, and it self-scores that at 0.55 confidence. Hardware price decay alone does not produce it; distillation, task specialization and algorithmic gains all have to stack. Plan against 10x per year at constant capability. The overhang points the same way: Astra sat internally for six months and the model behind this run is newer and externally unseen, so scaffolding built for today's weaknesses depreciates while evals, verifiers and clean data compound across model generations.

The loop closes at the row level

The compounding case has a physical address in your stack, and it is a table. A remediation bot writes a resolution. An LLM triage step writes a category. A ranking model writes an impression. Some downstream job then reads those rows as features or labels, and from that moment your offline metrics describe a system partly consuming its own past output, with no clean counterfactual behind the estimate. This is the pathology of a recommender trained on its own impression logs, restated in operations vocabulary by SRE Weekly's lead framing. Two instruments make it tractable, and both are unglamorous.

The precision math nobody watches

Positive predictive value tracks prevalence: PPV = p·TPR / (p·TPR + (1−p)·FPR). Hold true and false positive rates fixed, cut p by 2.7x, and a screener still parked on its old operating point returns roughly 2.7x more false positives per true positive. ROC and AUC barely budge, so nothing on the dashboard moves while decision quality degrades materially. Morning Brew's hiring numbers are the cleanest public instance, and the caveat travels with them: applications per opening is a ratio with an endogenous denominator, since years of post-pandemic downsizing shrink requisitions and inflate the ratio with no change in candidate behavior. The underlying data is also first-party, from Greenhouse's own customer base.

Rejects never generate labels

The documented harm from automated screening sits on the false-negative side, and it is unmeasurable by construction, because rejected items never produce an outcome. That is structurally identical to a credit model observing repayment only among approved applicants, and it takes the same fix: route 2 to 5% of algorithmic rejects through the full downstream pipeline and estimate the rate you were discarding. At sub-1% prevalence this needs real volume, so compute the sample size rather than picking a comfortable percentage.

Compounding shows up in joins too. At 99% per-source match precision, a name-based join across twenty sources leaves roughly an 18% chance of at least one mis-attributed record per subject, and none of that match uncertainty propagates into the feature the model consumes.

Your labelers are inside the loop

Morning Brew's other measurement is a calibration audit wearing a survey's clothes: reviewers self-report 60% accuracy at spotting machine-generated text and land near 25%, which on a roughly binary task is below chance, a 2.4x overconfidence gap. The survey publishes no sample size and no operationalization, so take the direction rather than the point estimate. The operating rule survives either reading: do not stack a miscalibrated human reviewer on top of a weak model and log it as human review. When the human signal is anti-correlated, the ensemble error rate rises while the process looks more careful.

The evidence is thinnest where the framing is best

SRE Weekly's issue contains no numbers at all, with no MTTR deltas, no defect rates and no sample sizes, and half its featured items are vendor-authored, with the skill-atrophy narrative conveniently favorable to incident-simulation vendors. Treat comprehension debt as a well-named hypothesis with no instrument attached, then attach one: autonomy rate per incident class, and median human diagnostic time on novel severe incidents. If both trend up together, the thesis holds in your data and quarterly game days are justified. If they do not, you saved the training budget.