The Product Desk

Only one cost term is settled

Your re-score needs one number: cost per successful task. That is tokens per attempt, times price, divided by success rate. Anthropic's release fixes the price term and shifts the token term. Simplifying AI worked the arithmetic. The per-token price fell 20%, so a net saving near 40% implies about 25% fewer tokens per task (0.8 × 0.75 ≈ 0.6). That is a derived figure, not an Anthropic disclosure. It still matters for your dashboards. A dashboard that compares models on dollars per million tokens will miss roughly half of this gain, and every future efficiency gain like it.

No vendor can give you the success-rate term. The SchrodingerRepo researchers found that on rewritten repos, 81.6–83.6% of the extra actions agents took were exploration, meaning the model searching through code it had never seen. That searching is what pushed input tokens up. Your private codebase triggers the same behavior, because no model has trained on it.

What unfamiliar code does to a business case

TheSequence ran an illustration with the paper's GPT-5.4-mini result. On a rewritten repo, input tokens rose 2.5x and first-try success fell from 46.8% to 35.6%. Together, that means roughly 3.3x the input-token cost per success. Even after a 40% price cut, you are near 2x what a leaderboard-based projection implies. The math mixes models and setups, so read it as a direction, not a forecast.

This risk doesn't hit all your planning documents equally. A cost baseline pulled from production already reflects your own code. A business case built from vendor demos or SWE-bench scores does not, so that is the one most likely to overstate ROI.

Both outlets say the headline figures are self-reported, but they differ on how fast to move. Simplifying AI treats Opus 5.5 as a model swap an existing Anthropic customer can make this week. It also suggests a routing pattern: default to Opus 5.5 and send only the hardest requests to Fable 5.1. That works because Anthropic claims Fable 5.1-level quality on “most tasks.” Simplifying AI adds that neither “most tasks” nor “typical workloads” is defined. TheSequence puts more weight on checking results against your own code before you commit.

A cost lever you control

Salesforce AI Research built JIT Mem, which stores raw records of past agent runs and assembles task-specific context only when a task needs it, instead of summarizing memory in advance. In the ALFWorld, WebShop and τ²-bench test environments, it cut input tokens 50.3–56.3% and steps 28.4–31.4%. It also beat memory systems that summarize in advance by up to 16.3 success points. Even an untrained Gemini curator scored 61.0, against 41.0 for SkillOS. These are benchmark environments, not production codebases. Still, JIT Mem targets the same exploration overhead the rewrite study exposed, and memory design is a decision your team owns.

The smart move

Build the eval before the business case. Draw 50–100 tasks from your own repos and workflows. Add variants that change how the code looks but not what it does, such as renamed namespaces and reordered files. They show how much a score depends on code the model has seen before. Then re-score the shelved backlog from those results, not from the launch post. Finally, clean up your own claims. Technical buyers can now easily challenge unqualified SWE-bench Verified scores in your PRDs or sales decks.

The spec finding aimed at your desk

Ayman Nadeem built Nuanced around persistent plans. He then found that long AI-generated specs added friction without improving understanding. His essay is titled “Plan Mode Is Dead,” but it argues against separating planning from building, not against planning itself. If your PRD process starts with “have the model draft the full spec,” this is a warning from someone who organized a whole product around plans: that step may be theater.

Fatih Arslan shows what survives. He keeps plans as living Markdown files in a shared Git repo, sorted into drafts, queued, active and completed folders. Long-running coordinator agents keep the index current. In effect, it is a Kanban board that humans and agents both read. Architecture Notes reconciles the two views: plans survive as a shared record of decisions and die as an upfront approval gate.

Why reviewing the finished run is too late

TheSequence's research roundup shows where oversight should move. Taste-Bench, from Microsoft and CityU Hong Kong, tests agents at decision forks. These are the moments where an agent picks an approach, expands scope or takes a destructive step. The longer the task, the harder the forks got. A related result is more encouraging. A distilled “advisor” model, meaning a smaller model trained to make judgment calls, advised an unchanged executor agent. With that advice, the executor's success rose from 14.6% to 33.7% on 41 held-out SWE-bench Pro tasks. Forty-one tasks is a small sample. The design lesson is that judgment at the fork is a separate component, and either a human or a small model can supply it.

WhatWorkedBench, from CMU and Tsinghua, adds a warning for products where agents analyze experiments. It covers 36 tasks and 1,248 configurations, and 35 involve sign-reversing interactions, where one setting flips the direction of another's effect. Agents found settings that worked but missed why they worked. The researchers paired agents with a Gaussian process, a statistical model that estimates effects from data. That raised effect recovery from 0.632 to 0.698 in one group of models and from 0.303 to 0.455 in another. If your product lets an agent explain A/B results or configuration choices, don't ship its “why” unaided.

What the oversight surface should contain

Brad Murry catalogued 13 orchestration patterns, ranging from pipelines and fan-out to supervisors and review loops. That gives your specs a shared vocabulary. Each pattern comes with use cases and failure modes. His design runs all of them on typed shared state with explicit transitions, which makes runs easier to inspect and recover. OpenMuse, an open-source agent covered by Simplifying AI, ships the checkpoint as a default: a separate gatekeeper asks for confirmation before any irreversible action. Simplifying AI notes that the gatekeeper protects users only if it correctly identifies which actions are irreversible.

Together, these point to three requirements most agent PRDs lack:

1. A named pattern with its failure modes, so reviewers know what breakage looks like.
2. A retry budget: the maximum attempts and spend before a human is pulled in. That is a product decision about cost and trust, not an implementation detail.
3. A decision record for each run: what the agent decided, which alternatives it rejected and which systems it touched, readable without opening a single diff.

What is known, and what is inference

A side-channel researcher opened Intel's bounty page and found no payout figures. The last archived version showing them is dated September 13. Intel has refused to explain the change to Risky Business or to the hardware news sites that spotted it. The program ran on Intigriti for more than a decade and added large bounties after Spectre and Meltdown. The link to AI-generated report volume is Catalin Cimpanu's inference, not Intel's statement. The pressure itself is documented. Over the past year several tech giants have said AI-found bug reports clog their programs and drain triage budgets. AI pen-testing products such as PortSwigger's Burp AT will push that volume higher.

Separate the pitch from the act. The pitch is noise reduction; the act is removing the cash. Academics working on side-channel and transient-execution attacks had reported payouts in the tens of thousands of dollars. Those go with the noise. Cimpanu predicts other companies will cite AI volume to justify the free-report model of two decades ago. Treat that as informed opinion, not a trend line.

Every queue runs on the same economics

Swap “bug report” for any unit a product handles one at a time: support tickets, feedback forms, app reviews, referral sign-ups, job applications. Each assumed real human effort per submission. When a submission costs almost nothing to produce, the cost moves to whoever reviews it. Cheaper models keep pushing that production cost down.

Identity is the same problem with bigger losses

The Fideuram fraud shows the bill. In February, fraudsters impersonated Intesa Sanpaolo CEO Carlo Messina in messages to Paolo Molesini, then chairman of Fideuram, the bank's private banking arm. A cloned lawyer's voice then got an executive to approve transfers to accounts in China and Hong Kong. More than half the money has been recovered. €36M is still missing. Analyst Hayden McKenzie separately tracked a North Korean IT worker cell operating behind a US front company called MageHire; at least three women appeared on camera in interviews and meetings while male developers did the work off-screen.

Simplifying AI flagged VoiceStudio, a free tool that runs entirely on local hardware and handles zero-shot voice cloning, dubbing and transcription in 646 languages. AudioSeal watermarking is on by default, and anyone with the code can switch it off. Cloned voices will circulate unwatermarked.

A submission, a voice or a face now proves only that someone had access to cheap generation tools. Trust has to rest on something costing the sender real effort or access: a proof others can reproduce, a reputation history, or a cryptographic confirmation over a separate channel.

The smart move

Measure before changing incentives. The useful number is the AI-generated share of each queue, per queue. Then filter without taxing honest senders: reproducible proof of concept, reputation weighting, duplicate clustering, AI pre-triage. The forcing question is whether a change raises the cost of submitting or the cost of being trusted. Intel raised neither. It cut the reward, which cuts the good reports too and may push the best researchers toward exploit brokers.