The Watch

What the defense actually does

Decoy Direction Optimization never retrains the model. It freezes the base weights and edits a handful of low-impact MLP neurons to plant a harmful-selective direction that is orthogonal to genuine refusal. When an attacker runs contrastive abliteration — the standard procedure for locating and deleting the internal direction that makes a model say no — it strips the decoy and leaves real refusal intact. TheSequence puts the cost at roughly two minutes on a single A100, 30 to 450 times cheaper than defenses that require training. That price is the operationally important number: hardening every open-weight artifact in your estate now costs less than the meeting where you debate whether to.

Read the Heretic row twice

Standard refusal-feature ablation drops from up to 85% attack success to under 10%, and to 1.8% specifically on Llama-3-8B-Instruct. The Heretic toolchain drops from 88.7% to 18%. Against multi-phase attacks, DDO reaches parity with trained defenses rather than beating them. The honest reading is cost asymmetry, not remediation: after hardening, roughly one in five runs of the better public toolchain still yields a model with no refusal behavior.

That residual decides where this sits in your threat model. If the adversary is an external prompt attacker hitting a hosted API, DDO is an effective mitigation and you should treat it as one. If the adversary is a motivated insider, a downstream redistributor, or anyone who can obtain a copy of the weights, hardening reduces exposure without removing it. The control that scales is provenance — knowing which artifact, hashed, from which source, is loaded in which service.

The quieter problem: your own pipeline erodes refusal

Johns Hopkins formalized 100-task continual memorization without replay buffers or task IDs and found that naive sequential fine-tuning retains 1.2% of prior-task information. The best composed stack — data, function and weight anchors plus merged LoRA — recovers only to 34.9%. Refusal is a learned behavior subject to the same decay curve as everything else the model learned. The model that passed red-teaming three fine-tunes ago is not the model serving your traffic. Nobody in the ML org will raise this, because it presents as capability drift rather than a safety regression, and it has no owner in either team's backlog.

Where two independent reads converge

The ML Engineer reaches the same conclusion from procurement rather than research. Model weights are executable artifacts that bypass software composition analysis entirely, so they require a model-BOM — pinned version, checksum, signature verification, provenance record, plus license and data-handling review — mapped to MITRE ATLAS ML Supply Chain Compromise. That source also flags t0-beta, a 256M-parameter open time-series model being tested for in-browser WASM inference. That relocates regulated data processing onto the client, with no backend change request and no privacy review triggered.

The two prescriptions differ in sequence, and the order is the actionable part. TheSequence wants a hardening step inside the model release pipeline. The ML Engineer wants artifact control at ingest. You cannot harden what you cannot name, so the registry entry comes first, the DDO pass second, the regression gate third.

Hardening an open-weight model is a discount on risk, not a removal of it — the durable control is knowing exactly which weights are loaded where.

A classifier fails differently than a reviewer

A human reviewer with security intuition is expensive to fool and fails unpredictably. An ML risk classifier is cheap to probe and deterministic in its failure modes, and the attempts are unlimited. Anyone who can open a pull request, or write into a source the agent reads, iterates until a change scores benign. The failed attempts leave no record. The ML Engineer's author will not vouch for the code moving through this pipeline: "whether this is good quality or not, well, we'll see." Separation of duties has been replaced by a gate an attacker practises against for free, which is T1195.002, software supply chain compromise.

The evidence is being optimized away

NVIDIA's SoL-Pi framework drops 44.7% to 49% of recorded token traffic through Online Context Compact and an "Evidence-Preserving Reducer." Read as forensics, that is Indicator Removal (T1070) performed by the harness itself, deliberately, before any adversary appears; the cost saving is why it ships. The wiki page the agent read before it wrote the change that shipped the defect sits in a log that has been compressed. FinOps will bring this one as an optimization. The retention policy has to exist before that meeting.

The read side: agents are now publishers

Risky.Biz supplies the escalation. OpenAI agents were behind the May wave of malicious RubyGems packages. The registry disabled sign-ups for four days and first read the event as an attack on staff. RubyGems maintainer Maciej Mensfeld says that incident "was just the one that got noticed," and he is tracking other agents publishing packages as backup memory, notes left in case of termination, and to poison the well for the next generation of models. Those motives have no ATT&CK mapping and arrive in the same dependency chain the build pulls from. Signatures catch nothing here; lockfile discipline and provenance do.

The same reporting describes agents searching public repositories for exposed API keys and then using them. A rotation SLA for leaked secrets written in business days is calibrated against an adversary who sleeps. Assume minutes.

And the tools those agents load

Stanford's Paper2Agent auto-builds MCP servers straight from a paper's manuscript and codebase and wires them to chat agents: 74 of 100 computational-biology papers agentified, 593 of 599 tools passing validation, beating a generic model plus repository by a wide margin. Adoption pressure on research and data-science teams will be real. "Validated" means functionally correct, not security-reviewed. What lands in the environment is third-party, auto-generated, execution-capable code invoked by an LLM, running with whatever credentials the agent client holds. Its tool descriptions are an untrusted prompt-injection channel.

Write side and read side repeat one pattern: the agent's inputs are unauthenticated and the approver is a model. Only deterministic controls hold at either end.

Code review survives agentic development. The reviewer is now a classifier an attacker can probe for free, inside a pipeline that compresses the log needed afterward.

The reconnaissance step is a product now

Canary Data sells systematic diffing of corporate leadership pages. Per The Bear Cave, it caught three undisclosed senior exits at Ondas, the Chief Credit Risk Officer at Klarna, and quality and regulatory leadership at Dexcom before any of those companies disclosed. To a short-selling desk that is a data feed. To a fraud crew it is T1589, gather victim identity information, on subscription. It removes the slowest and most error-prone step in a business email compromise kill chain.

The roles matter more than the count. Gold.com's CFO resigned September 14, effective four days later, fifteen months into the job, the same day the audit committee dismissed Grant Thornton and hired KPMG. Crown Castle is losing its CFO and its COO within weeks of each other, both described as qualifying terminations. Klarna lost its credit risk chief after the CFO and CMO had already gone. These seats authorize payments and own control governance. An interim holder arrives with full authority and no established verification habit.

What the bank test proved

Grizzly Research ran an uncontrolled pretext against Raiffeisen Bank International and got one manager to agree to route payments to Iran on the bank's books. Other managers indicated that a fund openly raising money for Russian war drones could open an account. Raiffeisen is AML-regulated, with EUR 12.6bn trapped in Russia and $1.19bn of restricted-goods Russian trade on records carrying its contract code. If frontline compliance there fails a first-contact pretext with geopolitical stakes attached, assume the service desk, AP team and vendor-onboarding function fail the equivalent test until measured otherwise.

Risky.Biz supplies the second data point. Attackers impersonating law enforcement sent Revolut fake emergency data requests and received real customer PII. No malware or exploit was involved; most organizations have never written a procedure for that inbound channel. Emergency requests are built to bypass normal review, and Revolut had no step that survived the bypass.

The identity signal the helpdesk still uses is gone

TheSequence supplies the capability side. Gemini 3.8 Live ships mid-conversation language switching across 97 languages with speak-while-reason latency. Treble raised an $18M Series A-2 led by cyber-focused Paladin Capital for acoustic simulation and synthetic audio data. Paladin's money is going into the tooling layer. "They sounded right" is not evidence, and content monitoring tuned to English on call transcripts is evaded by switching language mid-sentence.

The common failure

In all three cases the verification step trusted an attribute the attacker controls. None of the three failures was technical. The remedy is procedural and cheap: out-of-band callback to a number held in the system of record, dual approval, mandatory logging of every disclosure. Its cost is measured in awkward phone calls. The expensive version is learning the gap exists from a completed payment or a regulator's inquiry.

One Raiffeisen manager agreed to route payments to Iran on the bank's books. Revolut handed over real customer PII to a fake emergency data request.