Science & Analytics

The Scientist

The Signal

Anthropic killed the flat-rate Claude subscription this week.

Programmatic usage through the Agent SDK, GitHub Actions, and third-party tools now bills metered API credits at list price, which erases a 70–90% effective discount. ServiceNow burned its full-year Claude budget by May. OpenAI launched a 2-month-free Codex enterprise switch promo the same day.

In Play

  1. Anthropic's Pricing & Capacity Crisis

    Anthropic planned for 10x growth, hit 80x, and is leasing xAI's entire 220K-GPU Colossus 1 cluster to survive. Subscriptions are now metered at API rates. June 15 splits third-party tool credits with no rollover. ServiceNow burned its annual budget in 5 months. OpenAI is paying for your evaluation of the alternative.

    Ask Clarity
  2. 59% Agentic Token Share — Your Eval Harness Is Measuring the Minority

    Vercel's AI Gateway shows 59% of production tokens are now multi-turn agentic workloads. Anthropic captures 61% of spend via Opus while Google captures 38% of volume via Flash. Cost models built on single-turn assumptions are ~5x wrong on agentic spend. No vendor loyalty observed — teams route on task, not brand.

    Ask Clarity
  3. AI Cyber Capability Crosses Full-Takeover Threshold

    Anthropic's Mythos cleared both UK AISI simulated attack ranges — the first model to complete full network takeover end-to-end. GPT-5.5-cyber cleared one of two. Google confirmed AI-built offensive tooling observed in the wild. AISI is already building harder tests because the current ladder is saturating.

    Ask Clarity
  4. GPU Supply Crunch: 4:1 Demand Ratio, Neoclouds Sold Out

    Nebius reports 4+ customers per GPU brought online, 684% YoY revenue growth, and $3–3.4B 2026 guidance from a $530M base. Cerebras IPO'd at $56B with a $20B OpenAI commitment. Cisco AI orders jumping $5B→$9B. Memory hardware shortage now the next bottleneck after GPUs.

    Ask Clarity
  5. Training Efficiency: 3 Papers That Change Unit Economics

    Nous TST reports 2–3x wall-clock speedup at matched FLOPs with no inference architecture change (validated to 10B). NVIDIA Star Elastic claims 360x cheaper model-family derivation. Datology beat InternVL3.5-2B by 10 points at 17x less compute via data curation. The marginal dollar in training is moving from compute to curation and recipe.

    Ask Clarity

Deep Dives

Anthropic's 80x Miss: Your Claude Budget Is Already Wrong

What Happened

Three changes this week make every pre-May-2026 Claude cost model unusable. First, Anthropic converted paid subscriptions into dollar-matched API credits, which ends the implicit 70–90% discount developers were running through Agent SDK, claude-p, GitHub Actions, and third-party harnesses on Max plans. Second, Dario Amodei said at Code with Claude that Anthropic planned for 10x growth and hit 80x in revenue and usage. Third, they are leasing xAI's entire Colossus 1 cluster, 220,000+ NVIDIA GPUs across H100, H200, and GB200, from a CEO who called them 'misanthropic and evil' three months ago.

Capacity is the binding constraint. ServiceNow's CDIO confirmed they burned through the full-year Claude budget by May after price hikes hit a platform with no native per-user consumption telemetry.


The Pricing Change, Decoded

SurfaceBeforeAfter (June 15)
Agent SDK / claude-p / GitHub ActionsFlat subscription, unlimitedDollar-matched API credits, metered
Third-party tools (Zed, Conductor, OpenCode, T3)Bundled in planSeparate credit bucket, no rollover, overflow at API rates
Claude Code (Pro/Max/Team)5-hour limit, peak throttledLimits doubled, peak throttle removed
Opus API rate limitsSqueezed during crunch'Substantially raised'

Anthropic has hired a CFO and is targeting an October IPO. Margin-per-token is now a board-level metric. The subsidy regime is structurally over, not paused.


The Counter-Offensive

OpenAI shipped a 2-month-free Codex enterprise switch promo the same day. Ramp's April data had Anthropic at 34.4% versus OpenAI's 32.3% in business adoption, the first lead change. The thing that number doesn't tell you is durability under repricing, which is exactly what OpenAI is testing.

Any benchmark you ran between mid-April and now measured degraded-capacity Claude, not representative Claude. Re-baseline after the Colossus integration lands, not before.

What To Do

The immediate work is reconciliation: audit every Claude-backed workload and project token burn against the new credit cap. The structural work is routing: a provider-abstraction layer (LiteLLM, Portkey, or in-house) that makes vendor swaps a config change. The arbitrage work is the OpenAI promo. Use it as a free head-to-head window, but instrument with trajectory-level metrics. Pass@1 will not tell you which model your agents actually finish tasks on.

Anthropic provides no native per-user usage telemetry and no SLAs. Gateway-level logging with tenant, user, and feature tagging is now mandatory infrastructure. The vendor has offloaded that work to the customer, explicitly.

What to do

  1. Reconcile every Claude-backed workload (Agent SDK, GitHub Actions, batch evals) against new credit cap; flag jobs that will exhaust credits before month-end

  2. Deploy an LLM gateway with per-user, per-feature tagging and daily token budget alerts in front of all Claude traffic

  3. Run a 2-month Codex evaluation under OpenAI's enterprise switch promo with matched prompts and tool schemas

  4. Avoid locking into annual Anthropic contracts until post-Colossus integration stability is observable (likely Q3)

59% Agentic: Your Eval Harness and Cost Model Are Scoring the Minority

The Production Reality

Vercel's AI Gateway production index covers 200,000 teams over 7 months. Agentic workloads now sit at 59% of all token volume, up from under 20% six months ago. The spend-vs-volume split is the interesting cut: Anthropic captures 61% of spend, mostly Opus on reasoning and planning nodes, while Google captures 38% of volume, mostly Flash on throughput work. The data shows no vendor loyalty. Teams route on task type.

This is not a forecast. It is present-tense production telemetry, and most eval harnesses were not built for it.


Why Your Cost Model Is 5x Wrong

Single-turn cost models assumed input-to-output ratios around 3:1. Agentic traces run closer to 15:1 on input, with heavy cache reuse on some providers and none on others. A forecast built on last year's ratio is off by roughly five times on spend. The error is not symmetric across vendors, which is what makes it expensive to ignore.

ProviderGateway PositionPricing PostureImplied Role
Anthropic61% of spendPremiumReasoning / planning nodes
Google38% of volumeAggressive / free tiersHigh-throughput utility calls
OpenAIFast-growing shareMid-tierMixed; share spiking post-model-update
DeepSeek V4 ProEmerging$2.25/task (FlowGraph)Cost/capability sweet spot

Abridge's production architecture at 80M+ clinical conversations is the cleanest validation at scale: cheap fast model triages, expensive model reasons only when called, 5–10x cost reduction versus routing everything to a frontier model. Glean's benchmark claims MCP uses 30% more tokens than a retrieval-tuned knowledge graph. The number is vendor-published with no methodology, so treat it as directional. The failure mode it points at — verbose tool outputs bloating context — is well-documented elsewhere.


The Eval Gap

If 59% of your tokens are agentic but 100% of your evals are single-turn, you're flying instruments-out.

Standard harnesses score single-turn responses against reference answers. That was the right instrument in 2023. It does not measure the 59% of traffic that is multi-step tool loops with retries, where a planner burns 40K tokens arguing with itself before landing on the right answer. Final-answer accuracy is 90%+ in both regimes. The bill is where they diverge, and the bill is the bottleneck nobody is benchmarking.

Microsoft's MDASH (100+ agents) beat Anthropic's Mythos on CyberGym by decomposing into scan, debate, and exploit stages. The CyberGym result is consistent with multi-agent ensembles outperforming monolithic models on verifiable tasks. Given the numbers, I expect about half the reported lift to survive on production traffic once retries, tool-call failures, and inference cost are accounted for. Half the lift is still worth the migration. A quarter is not.

What to do

  1. Add trajectory-level metrics to eval harness: tool-call precision/recall, steps-to-completion, cost-per-successful-task, recovery-from-error rate

  2. Instrument per-node token cost in agent pipelines and route utility calls (summarization, extraction, rewriting) to Flash/Haiku-class models

  3. Run a 1-hour spike comparing MCP/tool-calling token overhead vs. a retrieval-first baseline on 100 production traces

  4. Add model-routing abstraction layer if not present; ensure every call site can swap providers without code changes

AI Offensive Cyber Hit Full-Takeover — Your Release Gate Needs a New Tier

The Capability Jump

The UK AI Security Institute ran the newest Anthropic Mythos and OpenAI GPT-5.5-cyber against autonomous cyber-offense tasks. Both completed full network takeovers in controlled environments, one tier above the prior Mythos generation, which capped at 'advanced persistence.' Mythos cleared both of AISI's hardest tests. GPT-5.5-cyber cleared one. AISI is already building harder benchmarks because the current ladder is saturating, which is the polite way of saying the eval no longer measures the bottleneck.

Separately, Google's threat-intel team observed a hacking group using LLMs to build a cybercrime tool in the wild. First detected incident of its class. The risk moved from tabletop to production event.


Why This Matters for Model Selection

Refusal rates and prompt-injection catch rates do not measure end-to-end attack-chain completion. The AISI result says the eval rubric needs stages: recon → initial access → lateral movement → persistence → exfil. A model that passes a jailbreak suite but chains exploits in production is a deployment risk the current harness cannot see.

ModelAISI Cyber TierHardest TestsRelease Posture
Mythos (new)Full network takeover2 of 2 clearedGated — select enterprises + gov
GPT-5.5-cyberFull network takeover1 of 2 clearedGated — handful of companies
Mythos (prior gen)Advanced persistenceBroader availability

On practical outcomes, the harness dominates the model. Mozilla wrapped a custom agentic harness around existing fuzzing infrastructure and surfaced 271 Firefox bugs with Mythos. Daniel Stenberg pointed the same model at curl and got 1 CVE with 4 false positives. Same weights, 271:1 yield difference. The variable that moved was the scaffolding: reproducible test cases, ephemeral VMs, integration into existing signal pipelines. That correlates with throughput. It is also the cleanest causal story available without a controlled run.

Vulnerability discovery moved from human-weeks to model-minutes. If patch SLAs are not benchmarked against inference time, the defense is tuned to last year's threat model.

Implications for Your Stack

For teams shipping agentic systems with tool access, two obligations follow. First, red-team suites built against GPT-4-era assumptions will produce false negatives against Mythos-class attackers. Run a fresh adversarial spike with a frontier model against internal services and measure time-to-first-exploit. Second, production telemetry needs agent-trajectory features. Log tool-call sequences, detect recon → lateral movement → persistence patterns, and alert on graph anomalies rather than prompt-level filters.

Palo Alto's AI-driven scanning surfaced serious vulnerabilities across 130+ products. Microsoft's MDASH shipped 16 real Windows fixes in May Patch Tuesday. The thing these numbers don't tell you is the false-positive cost on the defender side, but the unit economics of automated bug discovery have crossed the threshold where running is cheaper than not running.

What to do

  1. Add staged cyber-capability eval (recon, initial access, lateral movement, persistence) to your model release gate for any agent with tool/shell access

  2. Run a red-team spike using Claude Mythos Preview or GPT-5.5 against internal services; measure time-to-first-exploit vs. human baseline

  3. Instrument agent-trajectory features in production telemetry: tool-call graph anomalies and egress patterns, not just input-side filters

  4. Compress critical-patch SLA for any system reachable by LLM agents; expect sustained CVE volume uplift as AI-assisted scanning scales

Three Training Efficiency Breakthroughs That Change Your Q3 Compute Math

The Papers

Three research drops landed the same week. Each one nudges unit economics in a direction that matters if you are running training or distillation this quarter.

WorkClaimScale ValidatedInference ImpactReplication Risk
Nous TST2–3x wall-clock at matched FLOPs270M → 10B-A1B MoENone — no architecture changeMedium; single-source, clean claim
NVIDIA Star Elastic360x cheaper model-family derivation; 7x vs SOTA compressionNot specifiedProduces family of sizes from one runHigh; lab-reported headline number
Datology VLM curation+11.7 pts on 20 VLM benchmarks; 17x less compute2B and 4BLower response FLOPs — real serving winMedium; benchmark-selection risk

What Transfers

Token Superposition Training (TST) is the one to spike first. It is a pretraining recipe change with no inference-side downstream. If it replicates, it is a free 2–3x on wall-clock. The mechanism is architectural during training only; at serving time the model looks identical to a standard transformer. Validated from 270M to 10B params with an A1B MoE variant, which is a wider range than most recipe papers bother with.

Star Elastic's 360x is the kind of claim that always shrinks under independent eval. Given the numbers in the paper, I expect roughly an order-of-magnitude haircut on replication. Even a 30x hold would restructure how teams produce model-size tiers for deployment, because one post-training run producing a family of sizes eliminates the current pattern of training multiple checkpoints or running expensive compression passes per target.

Datology's result is the clearest evidence this year that the marginal dollar in VLM training has moved from compute to curation. Beating InternVL3.5-2B by ~10 points while using 17x less training compute, and producing a near-frontier 4B with 3.3x lower response FLOPs than Qwen3-VL-4B, is a serving-cost story as much as a training one. The thing the headline number does not tell you is which slices the 10-point gap concentrates in. Read the breakdown before you migrate.

The marginal dollar in training moved from compute to recipe and curation. If you're still scaling FLOPS before optimizing data, you're paying a 17x tax that Datology just quantified.

Practical Sequencing

TST is the lowest-risk experiment. Spike a 1B continued-pretraining run against a matched-FLOP baseline. If wall-clock comes in at even 1.6x with no val-loss regression, it pays for itself on the next full run. Star Elastic's value depends on whether you need to produce multiple serving sizes from a single training investment. If you do, the ROI is obvious at 30x and overwhelming at 360x. Datology's lesson is immediate and does not require reproducing the paper: audit your training data curation pipeline before adding more compute.

All three converge on the same point. The 4:1 GPU demand-to-supply ratio reported by Nebius makes efficiency recipes the highest-leverage alternative to fighting for more hardware.

What to do

  1. Spike Token Superposition Training on a 1B continued-pretraining run against a matched-FLOP baseline this quarter

  2. Audit VLM training data curation pipeline quality before next compute scale-up

  3. Evaluate Star Elastic once the paper publishes — flag if you produce 3+ model-size tiers from the same base

  4. Lock H2 2026 GPU reservations across 2+ providers before quarterly sellouts tighten further

The bottom line

Anthropic hit 80x growth on 10x capacity planning, killed the flat-rate Claude subsidy, and is leasing 220,000 GPUs from a competitor to keep the lights on — while 59% of production tokens are now agentic workloads that your single-turn eval harness doesn't measure and your cost model underestimates by 5x. Re-price your Claude budget this week, rebuild your eval around trajectories this sprint, and add a provider-routing layer before June 15 or you're making a $30B vendor's IPO margin target a line item on your quarterly invoice.