The Scientist

What Changed This Week

Three distinct Anthropic moves converged into a single cost event for any team running Claude programmatically:

1. Programmatic usage is now metered. Claude subscriptions convert to dollar-matched API credits across Agent SDK, claude-p, GitHub Actions, and third-party harnesses. The implicit 70-90% effective discount that alt-harness users had been getting is dead.
2. The 80x capacity admission. Dario Amodei conceded at Code with Claude on May 6 that Anthropic planned for 10x growth and hit 80x. The emergency patch: leasing xAI's entire Colossus 1 cluster (220,000+ GPUs) spanning H100, H200, and GB200.
3. June 15 third-party tool split. Starting in 30 days, Claude usage through Conductor, Zed, OpenCode, and T3 Code gets a separate credit bucket. No subsidized tokens, no rollover, overflow bills at API rates.

The Cross-Source Pattern

Multiple independent sources confirm this is a coordinated pricing tightening ahead of Anthropic's October IPO. ServiceNow's CDIO has already burned the full-year Claude budget by May. National Life Group's CIO publicly called Claude 'not great for companies' wanting per-user monitoring. Anthropic provides no native per-user telemetry and no SLAs — unusual for a dependency on the critical path of production features.

The vendor that captures 34.4% of enterprise spend cannot tell customers which user burned the tokens.

Meanwhile, OpenAI dropped a 2-month-free Codex enterprise switch promo the same day Anthropic metered usage. Ramp's April data shows Anthropic edging OpenAI 34.4% vs 32.3% — the first lead change. This is OpenAI pricing a counter-offensive against the exact developers Anthropic just alienated.

What Benchmarks From Before May 7 Are Now Stale

Any architectural decision made against April numbers — aggressive caching, prompt compression, provider migration — is calibrated to conditions that no longer exist.

The Practical Implications

Token consumption in agentic workflows is non-linear. A reflection loop or tool-use chain can 10x spend per task without proportional quality gain. With no native cost attribution at the user or prompt level, the overage shows up the way it showed up at ServiceNow — after the money is gone. The workaround is entirely on the customer: gateway-level logging with per-tenant tagging, daily budget alerts, and hard caps per feature.

The Production Reality

Vercel's AI Gateway production index — the first multi-tenant usage snapshot this quarter — puts agentic workloads at 59% of all token volume across 200,000 teams. Six months ago it was under 20%. That composition shift is faster than the completion-to-chat transition, and most production stacks were architected before it started.

The spend-versus-volume split is the routing signal. Anthropic captures 61% of dollars through Opus on reasoning and planning nodes. Google captures 38% of tokens through Flash on throughput and utility calls. Expensive models plan, cheap models fan out. Teams without a tiered router are paying Opus rates for work Flash does at 5-10x lower cost.

Why Single-Turn Evals Are Lying

Most agent eval harnesses still score a single response against a reference answer. That was the right call in 2023. With 59% of traffic now multi-step tool loops, three things break:

• Cost models break. Input-to-output ratios on agentic traces moved from roughly 3:1 to roughly 15:1. A forecast built on last year's ratio is off by about 5x on spend.
• Accuracy masks cost paths. A planner that burns 40,000 tokens arguing with itself and then gives up can still post 90%+ final-answer accuracy. The thing this doesn't tell you is where the bill lives.
• Tool-call reliability is invisible. An end-state-only score cannot distinguish a clean first attempt from a 3-retry recovery, and the production bottleneck is the second one.

Abridge's production architecture across 80M+ clinical conversations is the cleanest validation of the alternative: constellation-of-models with fast/slow routing, LLM judges calibrated against human-annotated ground truth, and memory externalized from weights into event-driven stores.

If 59% of your tokens are agentic but 100% of your evals are single-turn, you're flying instruments-out.

The Duolingo Anchor

Duolingo disclosed a ~20% unusable rate for AI-generated content at scale. Production quality numbers at that resolution are rare. They also reversed the blanket 'evaluate all employees on AI usage' policy after observing performative adoption with no productivity lift. Both points calibrate expectations. Well-resourced teams ship 1-in-5 outputs that need human rework, and token-usage-as-KPI is a Goodhart trap.

What the Eval Harness Needs

The Capability Threshold

Anthropic's Claude Mythos Preview is the first model to clear both UK AISI simulated attack ranges, a discrete ladder running from 'advanced persistence' to 'full network takeover.' The prior Mythos generation topped out one tier below. GPT-5.5-cyber cleared one of the two. AISI is already building harder tests because the current ones are saturating, which is the more informative signal.

This is not a benchmark delta. It is a discrete unlock: a task no prior model could complete end-to-end is now routinely clearable. Congress is pressing Anthropic through closed-door briefings, and the reported center of gravity is shifting from CISA (defensive) to NSA (offensive/intelligence).

The 271:1 Harness Result

Two teams ran Claude Mythos against large C codebases in the same month. The results diverge by two orders of magnitude:

Daniel Stenberg's verdict on the curl result: 'primarily marketing.' Mozilla's security team is integrating it into CI for all landing patches. Same model, same weights. The harness is the variable that moved.

This lines up with what Microsoft's MDASH showed separately: 16 real Windows vulnerabilities patched in May Patch Tuesday, found by a multi-model ensemble system. The pattern is that value accrues to the Cartesian product of how you probe × how you mutate × how you measure, not to the weights alone.

When a frontier model yields 271 bugs for one team and 1 CVE for another against the same language, the harness is the product, not the model.

Offensive AI Is Now a Detected Incident Class

Google's threat intelligence team identified a hacking group actively using LLMs to build cybercrime tooling, the first production-grade detection behind the post-Mythos misuse concerns. Palo Alto's AI-driven scanning surfaced serious vulnerabilities across 130+ products. The attacker economics have shifted. Inference is cheap, orchestration is cheap, and the expensive line item was always the operator. The model replaced the operator.

For teams shipping coding agents, the implications are concrete:

• Refusal-rate harnesses measure the wrong bottleneck. The thing a refusal rate doesn't tell you is whether the model can execute the chain. A staged rubric (recon, initial access, lateral movement, persistence, exfil) run against every model upgrade is the closer match.
• Agent trajectory telemetry is security telemetry. If a frontier model can execute a takeover in a lab, misuse attempts in production chain tool calls the same way. Log agent action sequences and train a lightweight classifier on known-bad trajectories.
• Patch SLAs need to benchmark against inference time, not human-weeks. Vulnerability discovery that used to take teams months now takes model-minutes.

Three CVEs That Chain Into Training Data Corruption

A new batch of critical CVEs lands on the exact infrastructure most ML teams run in production. These are not generic patch advisories. They target the data layer, which is the trust boundary between what a model trains on and what an attacker can manipulate.

The Attack Chain

1. Apache Polaris (CVE-2026-42809/10/11, CVSS 9.9) — Credential-broadening bugs let an attacker with limited catalog access escalate to full S3/GCS credential exposure, enabling cross-tenant data access.
2. Apache Iceberg (CVE-2026-42812, CVSS 9.9) — An attacker with table-write permission can redirect metadata pointers to an attacker-controlled S3 prefix, so the next query reads poisoned Parquet and the next training run ingests silently corrupted features.
3. Argo CD (CVE-2026-42880, CVSS 9.6) — Read-only users can extract plaintext Kubernetes Secrets. For teams running model services via Argo CD, that means model-registry tokens, HuggingFace PATs, and cloud credentials.

The combined path: compromised analyst notebook, Polaris credential escalation, Iceberg metadata redirect, poisoned training data. The failure mode is silent. Default lakehouse logging covers row changes, not pointer changes, so a row-count and schema check passes cleanly while the bytes have moved underneath.

Additional ML-Stack Vectors

The PraisonAI four-hour exploitation window resets the floor. Agent frameworks, model-serving gateways, and workflow orchestrators all shipped fast, and are now receiving the security attention web frameworks got a decade ago. The thing the CVSS score doesn't measure is blast radius inside an ML platform: a read-only Argo CD bug is a 9.6, but for a team that stores HuggingFace PATs as Secrets, the practical impact is the model registry.

If the reference architecture runs Iceberg, Polaris, Argo CD, or any modern orchestrator, there is patching homework before the next experiment and credential-rotation homework before the next sprint.