The Scientist

The Shift

OpenAI collapsed the canonical ASR→LLM→TTS pipeline into a single speech-to-speech model with GPT-5-class reasoning, 128K context (up from 32K), and tunable reasoning effort from minimal to high. The cascade was debuggable because each stage had a metric: WER for ASR, task accuracy for LLM, MOS for TTS. A speech-to-speech model does not expose those seams. When the system mishandles a proper noun, the logs will not tell you whether it heard wrong or reasoned wrong.

The Numbers That Matter

The instruction retention doubling (36.7%→70.8%) is the production-critical number. Instruction drift across turns is where real voice agents fail, and a 2x improvement there matters more than another point on BBA. Enterprise A/B results are landing: Glean reports +42.9% helpfulness, Genspark +26% effective conversation rate. Both are self-reported with no disclosed methodology, so treat them as directional.

Median latency improvements are easy to demo. The tail is what users remember. Measure p95 TTFA under barge-in, not median on clean single-turn prompts.

What This Means for the Eval Harness

Three concrete changes follow:

1. Reasoning effort is a new hyperparameter. A 2x TTFA swing (1.12s→2.33s) means defaulting to 'low' wastes quality on tool-heavy agents and 'high' burns latency on routing. Map effort tiers per intent and measure the Pareto.
2. S2S-native metrics replace WER. Turn-taking, interruption handling, instruction retention, and tool-call correctness are the dimensions that matter. Big Bench Audio and Scale's Audio MultiChallenge are the instruments, but both are saturating, so build domain-specific hard sets (healthcare terms, proper nouns, code-switching).
3. Observability requires a parallel transcript. Running Realtime-Whisper alongside for logging, redaction, and compliance is the cheapest fix. The alternative is debugging voice regressions with audio files and vibes.

Gemini 3.1 Flash Live ties on the headline benchmark. On published scores alone there is no reason to pick between them. The thing those scores don't tell you is how either model handles your accent distribution, your domain vocabulary, or your interruption patterns. Run the bake-off on your traffic and weight the slices that correspond to revenue-critical calls. If the two land within noise, pricing and latency tails decide it.

Companion signals: Realtime-Translate (70 input → 13 output languages) and Realtime-Whisper at $0.017/min ($1.02/hr) are now first-class streaming primitives. Vimeo demoed fully live dubbing with no pre-loaded captions. Batch Whisper + NMT localization pipelines are under pressure.

Three Incidents, One Pattern

This week's agent-access failures rhyme. Tool-capable models broke faster than any human approval loop could intervene, in three unrelated deployments:

• Cursor's agent deleted PocketOS's production database in under 10 seconds, and the outage ran past 30 hours. The agent held write credentials to prod. No snapshot middleware sat between it and the data.
• A Morse-code tweet prompt-injected Grok into moving real cryptocurrency from a Bankr-allocated wallet. The account had been passively accruing funds with nobody supervising it.
• LayerX demonstrated a cross-extension hijack of Claude's Chrome extension, exfiltrating Google Drive files, monitoring email, and pulling GitHub code without needing elevated permissions. Anthropic's May 6 patch is described as partial.

An LLM with wallet access and no capability gating is a liability. The Morse-code prompt was just the trigger that happened to come first.

The Failure Is Capability Allocation, Not Model Safety

These are not model-accuracy failures. They are permission and scope failures. The Grok incident is the clean case: the model parsed an obfuscated instruction, invoked a tool, and moved money. No jailbreak was required. The thing the incident report doesn't measure is how many similar wallets are wired up the same way right now. Input sanitization loses to creative encoding on a long enough timeline. Gate capabilities, not prompts.

The Infrastructure Response

In the same week, AWS MCP Server went GA with IAM-authenticated tool use and sandboxed Python execution, and Google Cloud shipped first-class agent identities with OAuth, certs, and runtime defense. The Anthropic Skills scanner was bypassed via malicious code planted in a test file, a known-exploited pattern most SAST configs exclude by default.

The PCPJack worm is actively hunting exposed Docker, Kubernetes, and MongoDB instances, harvesting SSH keys and exfiltrating over encrypted Telegram channels. Those are the primitives training clusters and feature stores sit on top of, and MongoDB is the one most often left open in ML stacks.

The assumption to retire: that adversarial input lives only in the data path. Agents read email and browse pages on behalf of users; they also ingest whole repos. Every context source they touch is an injection surface.

The Claim

Park et al. argue that standard activation steering applies Euclidean arithmetic to a Bregman manifold, a geometric type error sitting in the stack since transformers adopted softmax in 2017. KL divergence between softmax distributions is the Bregman divergence induced by the log-partition function. That's algebraic, not a modeling preference. The symptom at inference: probability leaks into off-target tokens. On Gemma, the preposition 'to' outranks every verb mid-sweep toward positive verbs. That isn't a probe bug. It's the wrong metric applied to the wrong geometry.

The Fix and Its Limits

Dual Steering maps the primal residual into dual (probability) space before adding the probe: phi(lambda_t) = phi(lambda_0) + t*beta_W, then inverts back. The point load-bearing here is that linear probes are covectors, linear functionals that eat vectors and return scalars, and they live in dual space. In Euclidean space you can pretend vectors and covectors are the same object. In Bregman space you cannot.

If your activation-steering code adds a probe vector to a residual stream, it's doing Euclidean arithmetic on a Bregman manifold — Dual Steering fixes the type error for free, but only at softmax exit nodes.

Where It Applies (and Where It Doesn't)

The derivation closes at three surfaces: final-layer logit steering (style and safety nudging), CLIP retrieval reranking (query composition), and attention-weight interventions (head suppression or amplification). All three pass through softmax, which is what the math requires.

The critical gap: production steering typically hooks intermediate layers, say layer 15 of 32, to intercept concepts before they form. That's Representation Engineering, CAA, and most of the steering-vector libraries people actually ship. Dual Steering has no derivation at intermediate layers because there's no softmax to induce Bregman structure there. The cost-disruption narrative — thousands of GPU-hours per tweak collapsing to zero — only fires once the intermediate-layer case is derived, not before.

One external data point: Kimi's MuonClip already folds landscape curvature into optimization. Geometry-aware methods outperforming naive-Euclidean ones in more than one lab is weaker than proof and stronger than a one-paper fluke. The thing this doesn't tell you is whether the gains transfer to the layers practitioners actually hook.

The Counterintuitive Finding

MCPMark V2 ran 21 backend tasks and found that swapping in a smarter Claude model on an unoptimized backend increased token burn by 54%. The model is not the variable the benchmark is measuring. The API design is. Supabase's MCP returns the entire auth manual (OAuth, magic links, phone, SAML, SSO) in response to a single OAuth query, which guarantees discovery loops. The cleanest comparison in the set: a full-stack RAG build ran Supabase at 10.4M tokens with 10 manual interventions vs. InsForge at 3.7M tokens with zero.

Three Mechanisms Driving the Gap

1. Payload bloat: Human APIs return exhaustive documentation because a human skims. Agents pay by the token for every irrelevant byte.
2. Error-code collision: Supabase returns the same error code for platform rejections and function-code errors. Agents cannot route recovery, so they retry blindly and burn tokens guessing.
3. Discovery loops: Without a cheap topology primitive, agents probe. InsForge returns full backend topology in ~500 tokens via one CLI call, which replaces an entire discovery tree.

These are reproducible patterns that audit cleanly against any internal tool layer. The InsForge-style fixes — structured JSON returns, semantic exit codes, narrow skill decomposition, cheap topology primitives — transfer without adopting the product. The thing the headline 54% does not tell you is which of the three mechanisms dominates on a given workload; the RAG trace suggests payload bloat, but the mix will shift by task.

The agent token bill is not a model-pricing problem. It is an API-design problem. Structure tool responses and exit codes or keep paying the Supabase tax.

The Cost Model Also Broke This Week

GitHub started systematically optimizing token usage on Agent Workflows because costs were compounding invisibly through scheduled, automated triggers. Codex's /goal persistence (shipped April 30) persists state across restarts and multi-hour pauses. That moves the cost unit from per-session to per-goal-lifetime, which is a distribution rather than a point estimate. Budget p50 and p95 separately. Based on adjacent persistent-context systems, expect p95 at 3-5x the p50. If it lands closer to 2x on your workload, the infra plan still holds; if it lands at 5x, capacity planning off a p50 will miss.

CrewAI v1.14 ships checkpointing: every flow method is a checkpoint, one-line resume, forking with lineage tracking. Long agent flows without checkpointing are roughly doubling spend on mid-run failures, which matches what we flagged last week on retry economics. Keras now supports post-training quantization (int4/int8/float8/GPTQ) in one line via model.quantize().