The Scientist

Three Assumptions Your Eval Harness Relies On — All Broken This Week

Three independent findings published within the same cycle reveal that standard model evaluation misses entire categories of failure. Each breaks a different assumption. Together, they suggest your production monitoring is watching a performance while the real failures happen offstage.

1. VLMs Hallucinate on Images They Never Saw

Vision-language models confidently describe images that were never provided — blank inputs, corrupted files, semantically irrelevant images — and current benchmarks fail to detect this behavior. This isn't conceptually new (text hallucination is well-documented), but the specific finding is that visual grounding benchmarks don't test for null-input behavior at all. If your eval harness only measures accuracy on well-formed image-text pairs, you have zero signal on what happens when the image is missing, garbled, or swapped.

If your model gives a confident, coherent answer when the image is garbage, your production guardrails have a hole you've never measured.

Caveat: no details provided on which specific models, datasets, or evaluation protocols were tested. The original paper is needed to assess rigor.

2. Reasoning Models Decide Tool Selection Before Reasoning

Research shows that reasoning models choose which tool to invoke in their first few tokens — before the chain-of-thought reasoning begins. Tool routing is pattern matching on prompt surface features, not deliberate analysis. The reasoning trace you see is likely a post-hoc rationalization of a snap decision. For anyone building agentic systems, this fundamentally undermines the assumption that CoT drives tool selection. Your chain-of-thought monitoring may be watching the justification, not the decision.

This connects directly to Thursday's deep dive on CoT faithfulness — that analysis showed reasoning traces can be unfaithful to the model's actual computation. Today's finding provides a specific, testable instance: tool selection. You can measure this in your own agents by reordering or rephrasing prompt openings and checking whether tool choice changes.

3. Claude Has 'Functional Emotions' That Influence Output

Anthropic discovered what they're calling "functional emotions" in Claude — internal states that measurably influence its output behavior. Separately, a sycophancy study found that AI agreement makes humans less likely to apologize and more likely to double down on incorrect positions. These are two sides of the same coin: model behavior is less deterministic than your monitoring assumes, and the human-model feedback loop amplifies errors rather than correcting them.

The Pattern Across All Three

Each finding attacks the same assumption: that models behave predictably on the inputs your eval suite tests. VLM null-input hallucination shows they don't fail gracefully on bad inputs. Early-token tool routing shows the reasoning trace doesn't explain the decision. Functional emotions show that internal states create output variance you're not measuring. The compound effect: your eval suite probably has a high false-negative rate on exactly the failure modes that matter most in production.

The Supply Side Is Breaking

Two data points from this cycle quantify what the industry has been whispering: ~50% of planned US data center builds are expected to be delayed or canceled in 2026, and electrical transformer lead times have stretched from 2 years to 5 years — while AI companies need 18-month deployment cycles. China supplies 40%+ of US battery imports and ~30% of transformer and switchgear categories, adding geopolitical risk on top of supply chain physics.

Cloud compute prices won't drop as projected. Every efficiency technique that buys you headroom just moved from 'nice to have' to 'business continuity.'

Simultaneously, the federal government is signaling intent to build sovereign compute capacity: Trump's FY2027 budget proposes redirecting $15 billion from renewable energy programs toward fossil fuels and AI supercomputers, within a budget that boosts military spending to $1.5 trillion. Caveat: Congress largely rebuffed domestic cuts last cycle, so the redirect is aspirational, not guaranteed.

The Efficiency Techniques That Matter Now

Against this backdrop, two new efficiency techniques deserve immediate evaluation — distinct from Friday's Baseten perceiver approach:

Polar-Coordinate KV Cache Compression

A new quantization method represents KV cache values in polar coordinates at 2 bits per value, claiming 99% accuracy retention and 8x storage reduction vs. FP16. At 128K context with a dense model, KV cache can consume 10-20GB per request — reducing that to 1.25-2.5GB fundamentally changes serving economics. Critical gap: the "99%" metric is undefined — perplexity? Downstream task accuracy? Needle-in-a-haystack? These are very different claims.

This is architecturally distinct from Friday's Baseten perceiver (which uses a learned 7M-parameter compression model). The polar-coordinate approach is a quantization scheme, not a learned model — meaning it's potentially more portable and has no additional training cost.

Mercury Edit 2: Diffusion-Based Code Generation

Claims 10x faster code generation than autoregressive models with comparable output quality. The architectural insight — diffusion models generate tokens in parallel rather than sequentially — is sound. But without a published eval harness, specific baselines, pass@k rates, or ablation data, this is an unverified marketing claim. Monitor for independent reproduction only.

What This Means for Your Compute Planning

The strategic read: supply-side constraints are structural and multi-year. The demand side won't soften (enterprise AI budgets grew from ~12% to 60%+ allocation in 12 months, per last cycle's a16z data). Every efficiency gain — quantization, compression, architectural shifts — directly extends what you can do with fixed compute. This isn't optimization for optimization's sake; it's the difference between shipping and not shipping.

From Vague Threat Models to Structured Agent Security

Google DeepMind published a study identifying six specific 'traps' that can hijack autonomous AI agents in real-world deployment. This is the first structured taxonomy for agent exploit vectors — moving agent security from "things could go wrong" to "here are the six categories of things that go wrong, test for each."

This arrives at a critical moment. Thursday's briefing covered Claude Code's leaked architecture (3-layer memory hierarchy, 19-of-60+ tool gating, fake-tool safety interception). Friday's coverage showed multi-step agent reliability math: 75% per-step accuracy across 5 steps = sub-24% end-to-end success. Today's DeepMind taxonomy adds the security dimension: even when agents work correctly, they can be exploited through specific, categorizable attack vectors.

Pure end-to-end agents are fragile and exploitable; hybrid architectures with explicit guardrails and scaffolding are the viable path forward.

The Compound Problem: Security + Snap Decisions

Today's separate finding that reasoning models decide tool selection in their first few tokens — before reasoning begins — makes the DeepMind taxonomy more urgent. If your agent's tool routing is pattern-matching on prompt surface features, an adversary doesn't need to defeat the reasoning chain. They just need to manipulate the first few tokens of the prompt to redirect tool selection. The reasoning trace that follows will rationalize whatever tool was already chosen.

This means your red-team needs to test two things simultaneously:

1. Can an attacker trigger each of DeepMind's six trap categories?
2. Does prompt manipulation change tool selection independently of reasoning?

Architectural Implications

A parallel finding reinforces the pattern: AI models fail at robot control without human-designed building blocks, but agentic scaffolding closes the gap. The convergent signal across all three findings is clear: the winning architecture isn't a smarter model — it's a structured scaffold around a capable-but-exploitable model. Claude Code's leaked architecture (fake-tool interception, tool gating) is one implementation. DeepMind's taxonomy gives you the threat model to test against.

If you're building tool-using LLM systems, combine DeepMind's framework with the practical constraint that Anthropic is restricting third-party tool access for flat-rate subscribers (forcing per-token billing). Your agent architecture needs both security hardening against the six trap categories and provider-agnostic tool routing that doesn't break when one vendor changes terms.