Science & Analytics

The Scientist

The Signal

Anthropic killed the flat-rate developer subsidy this week

OpenAI countered with a 2-month free Codex enterprise switch promo. ServiceNow already burned its full-year Claude budget by May. If you haven't reconciled projected token burn under the new metering regime before June 15, you're making a pricing decision by default that will show up as an overrun on the next invoice.

In Play

  1. Anthropic Pricing Shock Meets Capacity Crisis

    Anthropic metered all programmatic Claude usage at API rates, eliminated the 70-90% alt-harness subsidy, and revealed an 80x-vs-10x capacity miss. Emergency remedy: leasing xAI's 220K-GPU Colossus 1 cluster. ServiceNow burned a full-year budget by May. Opus 4.7 tripled image costs.

    Ask Clarity
  2. 59% Agentic Token Volume — Eval Harness Gap Widens

    Vercel's AI Gateway shows 59% of production tokens are now agentic multi-turn traces. Anthropic captures 61% of spend (Opus), Google 38% of volume (Flash). Single-turn eval harnesses measure the minority of traffic. Multi-agent decomposition (MDASH 100+ agents) beat monolithic models on CyberGym.

    Ask Clarity
  3. Training Efficiency: 2-360x Cost Reductions Validated

    Three independent results landed: Nous TST delivers 2-3x wall-clock at matched FLOPs with no inference change (270M→10B). Datology hit +11.7pts on VLM benchmarks at 17x less compute. NVIDIA Star Elastic claims 360x cheaper model-family derivation. DeepSeek V4 Pro: 77/100 FlowGraph at $2.25/task.

    Ask Clarity
  4. Lakehouse & Deployment Layer Now Under Critical CVEs

    Apache Iceberg (CVSS 9.9) lets attackers redirect table metadata to poisoned S3 paths. Polaris (CVSS 9.9) exposes cloud credentials cross-tenant. Argo CD (CVSS 9.6) leaks plaintext K8s Secrets. PraisonAI agent framework was weaponized in 4 hours. An 18-year NGINX RCE hits model-serving ingress.

    Ask Clarity
  5. Frontier Cyber Capability Crosses AISI Threshold

    Anthropic's Mythos is the first model to clear both UK AISI simulated attack ranges (full network takeover). GPT-5.5-cyber cleared one. AISI is already building harder tests because current ones are saturating. Mozilla's 271-vs-1 bug contrast confirms harness engineering dominates model choice by 50x+.

    Ask Clarity

Deep Dives

Anthropic's Metering Shock: Your Claude Cost Model Is Wrong in 30 Days

What Changed

Anthropic converted Claude subscriptions from flat-rate developer access into dollar-matched API credits. Every Agent SDK call, GitHub Action, claude-p pipeline, and third-party harness job (Conductor, Zed, OpenCode, T3 Code) now meters tokens at list price. The 70-90% effective discount alternative-harness users had been arbitraging is gone. Starting June 15, third-party tool usage gets a separate credit bucket. No subsidized tokens, no rollover, overflow at API rates.

Same day, Sam Altman posted a 2-month-free Codex enterprise switch promo. That timing is not coincidence. Ramp's April data shows Anthropic edging OpenAI 34.4% vs 32.3% in business spend, the first lead change. Anthropic has hired a CFO and is targeting an October IPO. Margin-per-token is now a board metric.


The Capacity Context

Dario Amodei conceded they planned for 10x growth and got 80x in revenue and usage. The fix is leasing xAI's entire Colossus 1 cluster — 220,000+ GPUs (H100, H200, GB200). Rate limits are loosening: Claude Code 5-hour limits doubling, peak-hours throttle removed, Opus API limits "substantially raised."

Any Claude benchmark from before May 7 is stale. The serving conditions your eval harness measured are about to shift again.

ServiceNow's CDIO has already burned through the full-year Claude budget by May. National Life Group's CIO calls Claude "great for consumer usage but not great for companies" wanting per-user monitoring. The thing the leaderboard score doesn't tell you: Anthropic provides no native per-user telemetry and no SLAs on latency or availability.


What Sources Agree On

Nine independent sources converged on the same conclusion: single-provider Claude dependency is now the highest-risk default on the table. The disagreement is on severity. Some frame it as a transient capacity issue that Colossus resolves in weeks. Others frame it as structural IPO-driven margin extraction. Both can be true simultaneously, and the cautious read is to plan for both.

The Contradiction Worth Noting

Anthropic is simultaneously the fastest-growing enterprise AI provider (ARR reportedly tripled from $9B to $30B+ in four months) and the one with the worst observability story for enterprise customers. Growth plus no telemetry plus no SLAs plus price increases produces the ServiceNow outcome at scale. The capacity numbers are correlated with the margin push. Causation runs through the IPO calendar.

What to do

  1. Audit every Claude-backed workload (Agent SDK, GitHub Actions, batch evals, third-party IDEs) and project token burn under metered pricing by end of this sprint

  2. Deploy an LLM gateway (LiteLLM/Portkey) with per-user, per-feature token tagging and daily budget alerts within 2 weeks

  3. Run OpenAI's 2-month Codex enterprise switch promo as a controlled A/B on your top 3 Claude workloads

  4. Avoid locking annual Anthropic contracts until post-Colossus serving stability is observable (target 6-8 weeks)

59% Agentic: Your Eval Harness Measures the Minority of Production Traffic

The Production Data

Vercel's AI Gateway telemetry, seven months across 200,000 teams, puts agentic workloads at 59% of all token volume. Anthropic takes 61% of spend through Opus. Google takes 38% of volume through Flash. Multi-provider is already the default; vendor loyalty does not show up in the data.

The spend-versus-volume gap is the architectural tell. expensive models handle planning and reasoning, cheap models do throughput work like retrieval rewriting, extraction, and classification nodes. Textbook tiered routing. Most teams still under-implement it.


The Eval Gap

Most eval harnesses still score single-turn responses against reference answers. When 59% of tokens are multi-turn tool-calling traces, that harness is measuring the minority of production traffic. The metrics that actually predict production behavior on agentic workloads:

  • Tool-call precision/recall. Whether the agent picked the right tool with the right arguments.
  • Steps-to-completion. Turns to resolve versus optimal.
  • Cost-per-successful-task. Not cost-per-token, which conflates cheap failures with expensive successes.
  • Recovery-from-error rate. Whether the agent recovers from a bad tool call without restarting.

Microsoft's MDASH (100+ agent ensemble) beat Anthropic's Mythos on CyberGym via a scan → adversarial debate → PoC construction decomposition. The pattern has real production value. The thing the paper does not tell you is which stage produced the lift, or what the ensemble cost relative to a single-agent baseline. Without an ablation, the result is suggestive, not decisive.


The MCP Convergence

SAP (€100M partner investment) and ServiceNow (Action Fabric) both shipped Knowledge Graph grounding + MCP-exposed workflows in the same week. TikTok shipped an MCP endpoint for external AI systems. The protocol is crossing from Anthropic-native to platform-standard. The enterprise agent stack is moving off chat-over-RAG toward governed execution layers.

Glean's benchmark claims off-the-shelf MCP uses 30% more tokens and loses 2.5x preference against an enterprise knowledge graph. Vendor-published, methodology not disclosed. Treat it as a hypothesis, not a result.

If 59% of tokens are agentic but 100% of evals are single-turn, the harness is measuring the wrong distribution. Fix the harness before swapping the model.

What to do

  1. Add trajectory-level metrics (tool-call F1, steps-to-completion, cost-per-successful-task, recovery rate) to eval harness this sprint

  2. Instrument per-node token cost in agent graphs and route utility calls (summarization, JSON extraction, query rewriting) to Flash/Haiku-class models

  3. Prototype a three-stage pipeline (generator → adversarial critic → verifier) on one auto-verifiable agent workload within 2 weeks

  4. Wrap top 2-3 internal tools as MCP servers for vendor-agnostic agent integration

Training Efficiency Breakthroughs: 2-17x Validated, 360x Claimed

Three Results That Change Unit Economics

WorkClaimScale ValidatedInference ImpactReplication Risk
Nous TST2-3x wall-clock at matched FLOPs270M → 10B-A1B MoENone — no architecture changeMedium; single-source, clean claim
Datology VLM curation+11.7 pts on 20 benchmarks; 17x less compute2B and 4B paramsLower response FLOPs — real serving winMedium; benchmark-selection risk
NVIDIA Star Elastic360x cheaper model-family derivationNot specifiedProduces family from one post-training runHigh; lab-reported headline number

Which One to Spike First

Token Superposition Training (TST) is the cleanest of the three. It's a pretraining recipe with no inference-side cost to amortize. If it replicates at even 1.6x on a 1B continued-pretraining run without a val-loss regression, it pays for itself on the next full training job. The matched-FLOPs framing is what matters here: speed is not being bought with quality, only with wall-clock.

Datology's result is the strongest evidence this year that the marginal dollar in VLM training has shifted from compute to curation. Beating InternVL3.5-2B by about ten points at 17x less compute, purely from data curation, changes how multimodal training budgets should be allocated. Their 4B model matches near-frontier at 3.3x lower response FLOPs than Qwen3-VL-4B. That is a serving-cost win, not just a training one.

Star Elastic's 360x is the kind of number that tends to shrink under independent evaluation. Even at 30x it would restructure how teams produce model-size tiers for deployment. Worth tracking. Not worth building around until someone replicates it on a workload that resembles production.


The Compute Supply Backdrop

Nebius reports 4+ customers competing for every GPU brought online, 684% YoY revenue growth, and 2026 guidance of $3-3.4B against $530M in 2025. Cisco's AI networking growth went from 5.3% to 12%, with 14% guidance. In that supply environment, every 2x in training efficiency is equivalent to doubling GPU allocation. The thing this doesn't tell you is which of these efficiency claims will survive contact with your own data distribution.

TST is a pretraining recipe with no inference-side cost — if it replicates at even 1.6x, it's free money on the next training run.

What to do

  1. Spike Token Superposition Training on a 1B continued-pretraining run against a matched-FLOPs baseline this quarter

  2. Lock H2 2026 GPU reservations across 2+ providers before quarterly sellouts tighten further

  3. Shift VLM training budget toward data curation: run a data-quality audit on your multimodal training corpus and allocate 30%+ of training effort to curation

  4. Pull SWE-ZERO-12M-trajectories (112B tokens, 12M trajectories, 3K repos) and stand up preprocessing pipeline for SFT or RM training

Your Lakehouse Just Became an Attack Surface: Iceberg, Polaris, and Argo CD

What's New This Cycle

The attack surface moved off model proxies (LiteLLM, covered Tuesday) and into the data and deployment layers most DS teams treat as trusted infrastructure. Three new critical CVEs landed this week:

  • Apache Iceberg CVE-2026-42812 (CVSS 9.9) — An attacker with table-write permission can redirect metadata to an attacker-controlled S3 prefix, so subsequent queries read poisoned Parquet and downstream training runs ingest silently corrupted features. Default lakehouse observability does not cover metadata pointer mutations.
  • Apache Polaris CVE-2026-42809/10/11 (CVSS 9.9) — Credential-broadening bugs expose S3/GCS credentials and enable cross-tenant access. Combined with the Iceberg bug, the path from "compromised analyst notebook" to "cross-tenant data theft" is plausible rather than theoretical.
  • Argo CD 3.2.x/3.3.x CVE-2026-42880 (CVSS 9.6) — Missing authorization lets read-only users extract plaintext Kubernetes Secrets. For teams running model promotion through Argo, every K8s Secret in reachable namespaces (registry tokens, HF PATs, DB passwords, cloud credentials) should be treated as disclosed.

The Training Data Poisoning Scenario

The Iceberg vulnerability is the one that matters most for data science specifically. The failure mode is not deletion. It is silent corruption: an attacker redirects table metadata to a poisoned prefix, and that propagates through the feature pipeline into training, with the resulting model shipping a subtle distribution shift that existing data quality monitors will not flag because the schema is unchanged and only the content is adversarial.

The thing most lakehouse observability does not tell you is whether metadata pointers mutated, because default logging covers row changes, not pointer changes.

Adjacent Threats

PraisonAI (multi-agent framework) was weaponized within 4 hours of CVE disclosure, which sets the floor for agent-framework patching at same-day. An 18-year NGINX RCE in the rewrite module affects every model-serving gateway behind NGINX, which is most of them. The Fragnesia kernel LPE, third in the Dirty-Frag class, targets multi-tenant training clusters for tenant escape.

Workflow orchestrators are also bleeding

n8n has SQLi (CVSS 9.8), Kestra ≤1.3.3 has SQLi (CVSS 9.8), and Spring Cloud Config has directory traversal (CVSS 9.1). These tools typically run with broad cloud credentials because they orchestrate everything, which is exactly why a CVSS 9.8 in the orchestrator does not stay in the orchestrator.

What to do

  1. Audit Iceberg/Polaris catalog configs this week: enforce explicit storage credential scoping and add write-path allowlisting for table metadata locations

  2. Patch Argo CD to ≥3.2.12 / ≥3.3.10 and rotate every Kubernetes Secret in namespaces it can read

  3. Patch NGINX across all inference gateways and audit rewrite-module usage in model routing configs

  4. Add metadata-pointer mutation monitoring to lakehouse observability (table-location audit log, write-path diff alerting)

The bottom line

Anthropic's flat-rate developer subsidy is dead, 59% of production tokens are agentic traces your single-turn eval harness doesn't measure, and three CVSS 9.9 vulnerabilities in the lakehouse layer just opened a path from 'compromised notebook' to 'poisoned training data' — reconcile your Claude token burn before June 15, rebuild the eval harness around trajectories, and patch Iceberg before the next training run ingests something an attacker wrote.