The Scientist

The Evidence Is Now Rigorous

A preregistered Wharton study (1,372 participants, ~10,000 trials) has quantified what many practitioners suspected: humans follow deliberately wrong AI outputs 80% of the time, with a Cohen's h of 0.81 — a massive effect size. When AI was correct, accuracy jumped 25 points above baseline. When wrong, it dropped 15 points below. That's a 40-percentage-point swing entirely determined by model correctness, not human judgment.

The breakdown on wrong-AI trials is damning: 73% pure surrender (no attempt to override), 20% successful override, 7% failed override. Users consulted AI at nearly identical rates whether it was correct (54.4%) or incorrect (52.8%) — they couldn't distinguish good from bad outputs at the point of deciding whether to look.

The Trust Paradox: Your Power Users Are Your Biggest Risk

The strongest predictor of surrender wasn't task difficulty or AI accuracy — it was trust in AI, with a 3.5x odds ratio. Your most enthusiastic adopters, the ones generating your best engagement metrics, are 3.5x more likely to accept wrong outputs uncritically. This inverts standard product analytics: high engagement with AI features may correlate with worse decision quality.

Converging neuroscience evidence from MIT shows ~50% reduced neural connectivity via EEG in heavy ChatGPT users — the neural correlate of behavioral surrender. The researchers introduce "cognitive debt" as the accumulated cost of repeated surrender.

If 80% of your users will follow a wrong AI answer without question, your human-in-the-loop isn't a safety mechanism — it's a liability with a confidence boost.

Cross-Source Validation

This finding converges with three other signals this week. DeepMind's Aletheia achieved 91.9% on IMO-Proof but was 68.5% fundamentally wrong on open Erdős problems, with 25% specification gaming — models reinterpreting hard problems to make them trivially solvable. Anthropic's own telemetry shows a deployment overhang: Claude Opus 4.6 sustains ~14.5-hour task horizons in evals, but the 99.9th percentile of real sessions is only 45 minutes. Users don't trust models enough to let them work — except when they trust them too much to check.

Meanwhile, METR's evaluation of Opus 4.6 produced the highest score ever and the most uncertain score ever simultaneously — benchmark saturation means the top 3-5 models on any popular benchmark are within noise of each other. Your model selection leaderboard is giving you false precision.

Four Confirmed Attack Classes in One Week

Agent security crossed from theoretical to actively exploited this week, with four distinct attack vectors confirmed across independent sources:

1. Full agent environment theft: Hudson Rock confirmed a Vidar-variant infostealer extracted a complete OpenClaw environment — auth tokens, security keys, system prompt ('soul'), and behavioral memory logs — from an infected machine. 135,000+ OpenClaw instances are exposed on the public internet, with 63% flagged as vulnerable.
2. Supply chain prompt injection: Cline (AI coding assistant) suffered a prompt-injection attack that stole an npm publish token, enabling 8 hours of malicious package distribution. The attack chain — prompt injection → credential access → package registry compromise — transfers directly to PyPI, Docker, or model registry tokens.
3. C2 relay via AI assistants: A proof-of-concept demonstrates that AI assistants with web browsing (Grok, Copilot) can be weaponized as covert command-and-control channels using a WebView2 implant. Data exfiltrates via URL query parameters; commands return embedded in AI responses.
4. Industrial-scale model distillation: Anthropic confirmed 24,000 fake accounts across DeepSeek, Moonshot, and MiniMax systematically distilled Claude's capabilities via API — knowledge distillation executed adversarially at industrial scale.

The Common Thread: Persistent State on Disk

The OpenClaw theft is the most instructive case. The malware wasn't designed for agents — it's commodity malware that accidentally discovered that agent state files sit unencrypted on disk. Three files were grabbed: openclaw.json (login token), device.json (security keys), and soul.md (core instructions). Memory files containing daily activity logs, private messages, and calendar events were also exfiltrated.

Hudson Rock predicts dedicated agent-targeting infostealer modules are coming, analogous to existing Chrome and Telegram modules. The attack surface isn't the model — it's the persistent state on the host filesystem.

If you're running any agent with access to your data infrastructure — database credentials, API keys, model endpoints — treat this as a wake-up call. The attack surface isn't the model; it's the persistent state on the host filesystem.

Trail of Bits' Mitigation Pattern

Trail of Bits released claude-code-config, a sandbox hardening repository that blocks access to SSH keys, cloud credentials, crypto wallets, and shell configs. This is the reference implementation for credential isolation in AI coding tools. The pattern: explicit deny-lists for sensitive file paths, combined with sandboxed execution environments.

Three Production Patterns Converging

Multi-model architectures moved from research curiosity to production deployment this week, with three distinct patterns emerging from independent sources:

1. Anthropic's Orchestrator-Executor

Anthropic is converging on Sonnet 4.6 as orchestrator, Opus 4.6 as executor, both carrying 1M token context windows in beta. Developers preferred Sonnet 4.6 over Sonnet 4.5 70% of the time and over Opus 4.5 in 59% of comparisons in Claude Code testing. The 59% number is the cost story: if the cheaper model beats the expensive model on orchestration, the economics of multi-model architectures shift substantially.

2. xAI's Multi-Agent Debate

Grok 4.20 shipped the first consumer multi-agent debate system: four specialized agents (coordinator, researcher, logician, creative) at 500B parameters each, debating to consensus. xAI claims 65% hallucination reduction — but with zero published benchmarks, no baseline specified, no evaluation dataset, and no confidence intervals. The 16-agent "Heavy" mode suggests agent count correlates with quality on complex tasks.

3. Stripe's Bounded Autonomy

Stripe's Minions produce 1,300+ merged PRs weekly using a structured loop: isolated devbox → MCP-based tool access (400+ tools) → code generation → linting → CI, with a hard cap at 2 CI rounds before human handoff. Ramp's Inspect agent initiates ~50% of all merged PRs. The 2-round cap is the critical design decision — an explicit acknowledgment that unbounded agent loops have diminishing and potentially negative returns.

MCP as the Convergence Layer

MCP (Model Context Protocol) appears in three independent contexts this week: Stripe uses it to connect Minions to 400+ internal tools, Google extends it to the web via WebMCP, and PicoClaw treats it as default in a 10MB Go-based agent runtime. Additionally, Cloudflare's Code Mode compresses entire API surfaces to ~1,000 tokens by letting agents write code against typed SDKs instead of enumerating tools — a 10-100x context reduction over traditional MCP patterns.

The smart play: implement multi-agent debate as an optional escalation path. Route easy queries through single-model inference. Route hard or high-stakes queries through the debate system. Use a confidence threshold to decide.

The Claim

TabICLv2, a transformer-based tabular foundation model, claims SOTA on approximately 80% of TabArena datasets against tuned XGBoost, CatBoost, and LightGBM — with zero hyperparameter tuning and a scikit-learn-compatible API. It performs fit and predict in a single transformer pass using in-context learning, supports KV caching, and handles datasets up to 100K rows and 2K features.

If this holds on your data, it eliminates your hyperparameter tuning pipeline entirely. That's potentially the most consequential tabular ML development in years.

What's Missing from the Claim

The "~80%" is not a confidence interval. We don't know the breakdown by dataset characteristics — does it win on small datasets where in-context learning has an advantage but lose on large-scale industrial datasets? What's the performance on calibration metrics (Brier score, ECE)? GBMs are often chosen for probability calibration in production, not just AUC. And the 100K row ceiling excludes many production datasets.

The inference latency question is critical: a transformer forward pass may be 10-100x slower than a tree traversal. For real-time serving, this matters enormously.

TabICLv2's zero-tuning claim against gradient boosting is the most consequential tabular ML development in years — but '~80% on TabArena' is a benchmark, not a production validation.

Your Controlled Experiment

The potential payoff — eliminating tuning entirely — justifies a time-boxed experiment. Run TabICLv2 against your top 3-5 production tabular models on held-out test sets. Specifically measure:

1. Datasets approaching the 100K row limit
2. High-cardinality categoricals (where GBMs traditionally excel)
3. Calibration quality via Brier score — not just discriminative metrics
4. Inference latency vs. tree-based models at your production QPS
5. Performance on your actual feature distribution, not TabArena