The Scientist

Three Providers, Zero Overlap at the Top

February 2026 delivered benchmark results that shatter any remaining case for a single-model strategy. The performance gaps are large enough to be operationally significant:

*Exact Sonnet 4.6 scores on OS World and SWE-Bench Verified described as "new records" without specific numbers.

SWE-Bench Is Dead — Build Your Own

Compounding the routing problem: SWE-Bench Verified is no longer reliable. Multiple sources confirm two compounding failures — flawed test cases that reject correct fixes, and training data contamination making exposure a significant scoring factor. OpenAI has officially abandoned it. If you're still using SWE-Bench scores to select coding models, you're optimizing for a broken signal.

Sonnet 4.6: The Cost-Performance Calculus Shifts

Sonnet 4.6 ships with a 1M-token context window (4x previous, currently in beta) at unchanged pricing. Early testers preferred it over Sonnet 4.5 ~70% of the time and over Opus 4.5 ~60% of the time. This fundamentally changes the RAG tradeoff — for document QA workloads where your corpus fits in context, you can potentially eliminate your entire chunking → embedding → vector DB → retrieval pipeline. Caveat: preference rates lack reported sample sizes or statistical significance. The 1M window is in beta — validate latency and accuracy at extreme context lengths before migrating production workloads.

The New Iteration Cadence

Sonnet 4.6 shipped 12 days after Opus 4.6. Gemini 3.1 Pro more than doubled its predecessor's ARC-AGI-2 score in a single generation. OpenAI is deprecating 5 models including GPT-5 and GPT-4.1. Quarterly model evaluation reviews are obsolete — you need automated eval triggers that fire within 48 hours of any frontier release.

The era of a single frontier model is over. The era of task-specific model routing has arrived.

The Attack Surface Is Bigger Than Rate Limiting

Anthropic's disclosure is the most detailed public account of systematic model extraction ever published. The numbers across 12+ sources are consistent: 24,000 fraudulent accounts, 16M+ exchanges, with MiniMax responsible for approximately 13 million (81%) of those exchanges alone. DeepSeek targeted logic and alignment, Moonshot AI targeted agentic reasoning and coding, and MiniMax targeted agentic coding.

The Target List Is Your Capability Priority Map

The most strategically valuable insight isn't the attack itself — it's what they targeted. The distillation campaigns focused on: agentic reasoning, tool use/orchestration, coding and data analysis, computer-use agent development, computer vision, and rubric-based grading for RL reward models. That last item is particularly significant — it suggests these labs are trying to bootstrap their own RLHF/RLAIF pipelines by extracting Claude's reward signal. This is the clearest public signal of which capabilities represent genuine technical moats.

Why Standard Defenses Failed

At ~667 conversations per account on average, individual account behavior was well within normal usage patterns. Standard per-account rate limiting completely missed this. The campaigns used "hydra cluster" proxy networks to bypass regional restrictions. The detection signal lives in cross-account coordination: query embedding similarity across accounts, temporal burst correlation, IP/proxy overlap, and account creation velocity. This is a graph problem — you need to model relationships between accounts, not just individual account behavior.

Sources Disagree on Detection Timing

Multiple sources note that 16M exchanges running before disclosure suggests either delayed detection or delayed public response. Anthropic hasn't disclosed their detection methodology — whether this was real-time behavioral analysis or forensic reconstruction matters enormously for whether this is a repeatable defense. Google reported a similar attack two weeks prior — over 100,000 prompts targeting Gemini — suggesting this is an industry-wide pattern, not an Anthropic-specific vulnerability.

If 16 million distillation queries can slip past Anthropic's defenses, your model API's biggest vulnerability isn't prompt injection — it's systematic capability extraction at industrial scale.

SAGE: Self-Aware Reasoning Termination

SAGE introduces a mechanism for reasoning models to self-determine when to stop thinking, eliminating redundant chain-of-thought tokens after reaching a correct conclusion. SAGE-RL distills these efficient patterns into standard pass@1 inference. The claim: accuracy improvement with fewer tokens across six math benchmarks. If you're running o1-class or DeepSeek-R1-class models and paying per-token, even modest reductions in average chain-of-thought length translate directly to cost savings.

Critical gap: no specific numbers are reported — we don't know if this is 5% or 50% token reduction. Results are math-only; generalization to code, multi-hop QA, or planning tasks is unvalidated. Treat as a research direction to prototype, not a proven technique.

The Recall Bottleneck: Your Fine-Tuning May Be Solving the Wrong Problem

Google's "Empty Shelves or Lost Keys?" paper makes a clean distinction: when a model gets a factual question wrong, is it because the fact was never encoded (empty shelf) or because the model can't access what it knows (lost keys)? The finding — that recall, not encoding, is the limiting factor — has immediate implications. If you've been fine-tuning models on domain knowledge and seeing diminishing returns, the knowledge may already be there. Your investment should shift toward inference-time retrieval mechanisms, better prompting, or architectural changes to attention and memory.

Diagnostic test: Take your worst-performing factual queries and test whether the model can answer them with heavy context priming (essentially giving it the answer in the prompt neighborhood). If accuracy jumps, your problem is recall, not knowledge — and your investment should go to retrieval infrastructure, not more fine-tuning data.

Embedding Quality: Local Neighborhoods, Not Global Cosine

EPFL's revision of the Platonic Representation Hypothesis shows that representation convergence happens in local neighborhood structure, not global embeddings. This means your embedding quality metric should be k-NN neighborhood preservation, not just cosine similarity. If you're evaluating embedding models for RAG or search, test local neighborhood consistency as your primary quality signal.

Masked Gradient Updates: A Cheap Stability Fix

Google's work on masked gradient updates in adaptive optimizers shows surprising effectiveness for training stability — a potentially cheap fix for loss spikes that doesn't require architectural changes. If you're training or fine-tuning large models, try masking out low-magnitude gradient components in Adam/AdamW and compare training curves before reaching for more expensive stability interventions.

Five New Architectures Worth Tracking

The Feature Gap Is Deliberate

Nearly two years after spending over $1 billion to acquire Tabular (the company behind Apache Iceberg), Databricks' managed Iceberg implementation is deliberately limited. The gap between open-source Iceberg and Databricks managed Iceberg is not a matter of catching up — it's architectural:

You must enable Predictive Optimization to use managed Iceberg at all. The platform's new guidance is blunt: "don't partition or sort or bucket your tables." Liquid Clustering, Predictive Optimization, and Photon replace every physical data modeling lever that data engineers previously controlled.

Why This Matters for ML Pipelines

Reproducibility risk: If your feature engineering pipelines depend on deterministic file layouts for reproducible training runs, Liquid Clustering's opaque layout management means you can't guarantee the same query over the same logical data will scan the same physical files across runs.

Multi-engine access: Many ML teams read training data through Spark but serve features through different engines (Trino, DuckDB, Presto). Managed Iceberg's Predictive Optimization requirement creates a Databricks dependency in your read path that undermines the multi-engine promise of open table formats.

The debugging problem: When your 4TB daily feature engineering job suddenly takes 3x longer because Liquid Clustering made a suboptimal layout decision, you have no manual override. No partitioning knobs, no compaction controls, no file layout inspection. You file a support ticket.

The Power User Signal

Practitioner Zach Wilson migrated 13,000 tables to AWS Glue Catalog instead of Unity Catalog after Databricks deprecated the Tabular metastore — a concrete data point that experienced users are routing around Databricks' governance layer to retain engine flexibility.

Important caveat: This analysis comes from a single practitioner's experience. There are no performance benchmarks comparing Liquid Clustering vs. manual partitioning, no TCO analysis, and no discussion of whether Predictive Optimization's decisions are inspectable. Before making architectural decisions, run your own benchmarks on your own workloads.

Databricks is trading your physical data modeling controls for platform-managed automation, and its managed Iceberg implementation strips the features that make Iceberg worth using.