The Scientist

Two Companies, One Lesson: Constrain or Fail

Amazon confirmed what many suspected: AI-generated code is causing production-scale failures. A nearly 6-hour retail outage and a 13-hour AWS disruption were linked to Amazon's own Kiro coding tool. The company convened senior engineers and now mandates senior sign-off on all AI-assisted code changes by junior and mid-level engineers. These aren't edge cases — Amazon internally described them as 'high blast radius' incidents where changes propagated widely across services.

Meanwhile, the New York Times demonstrated the opposite trajectory. By constraining AI to test generation only — read-only coverage reports, a hard rule against editing source code, mandatory human review — they raised average test coverage from 28% to 83% across six web projects with an estimated 70% reduction in effort.

Unconstrained AI coding is causing 13-hour outages at the world's largest cloud provider. Constrained AI coding is tripling test coverage at one of the world's largest publishers. The difference is three guardrails, not model capability.

The Benchmark-to-Production Gap Is the Same Gap You Already Know

Claude Opus 4.5 scored 92% on Stripe's 11-task benchmark for full-stack integration. That same week, Amazon's AI coding tool caused a 13-hour outage. This is the coding-agent version of your model hitting 0.95 AUC offline and then catastrophically failing on distribution-shifted production data. Stripe's 11-task sample is far too small for reliable agent comparison — no published confidence intervals, no ablation by task type, no variance across runs.

This corroborates the METR study (previously covered): ~50% of benchmark-passing AI-generated PRs are rejected by real maintainers from scikit-learn, Sphinx, and pytest. The failure modes that benchmarks miss — code quality, maintainability, interaction with adjacent systems — are exactly what caused Amazon's outages.

The Three Guardrails That Work

The NYT pattern is immediately transferable to ML pipeline code:

1. Scope restriction: AI generates tests only, never modifies code under test
2. Read-only observability: AI can see coverage reports but not alter reporting infrastructure
3. Mandatory human review: every generated artifact is reviewed before merge

For ML teams, map this to: point an AI coding agent at your feature pipeline with read-only access to coverage metrics, a constraint against editing source code, and senior sign-off. If you replicate even half of the 28% → 83% gain, you've dramatically reduced exposure to silent data corruption — the ML equivalent of Amazon's high-blast-radius failures.

Methodological caveat: NYT's results are self-reported, no holdout group, unclear project representativeness. Directionally strong, but treat the specific numbers as upper bounds.

New Tooling: Agent Safehouse

Agent Safehouse (1.3k GitHub stars, v0.3.1) provides deny-first sandboxing for macOS coding agents (Claude Code, Codex, Amp). It uses composable policy profiles to restrict what agents can access — critical if your repos contain training data paths, model weights, or cloud credentials. Caveat: built on macOS sandbox-exec which Apple has deprecated, so long-term viability is uncertain.

Nvidia Admitted GPUs Aren't Enough

At GTC 2026, Jensen Huang is expected to unveil a rack system combining Nvidia GPUs with Groq's Language Processing Unit (LPU) — 256 inference-specialized chips per rack — backed by a ~$20 billion licensing deal. OpenAI is reportedly the first buyer, to power its AI coding agent. This is not an OEM partnership. This is Nvidia integrating another company's processor into its own server architecture for the first time.

Nvidia paying $20B to license inference-specialized silicon is the loudest signal yet that GPU-only serving is a dead end at scale.

Architecture Details and Red Flags

The 256-LPU rack uses a different architecture from existing Nvidia racks. Critically, Intel processors manage inter-chip communication — not Nvidia's NVLink/NVSwitch. This is a telling detail: Nvidia's own high-speed interconnects don't yet integrate with the LPU. The first-gen system is a bridge, not an integrated solution.

The Samsung manufacturing wildcard is significant. Samsung's advanced node yields have historically lagged TSMC's. Nvidia plans to move back to TSMC for the Feynman-generation LPU fusion — telling you everything about their confidence in Samsung as a long-term partner.

What This Means for Your Serving Stack

You now have three inference-specialized hardware paths alongside GPU baselines:

1. Groq LPU — available now via GroqCloud API; Nvidia-integrated rack H2 2026
2. Cerebras via AWS — new partnership, cloud-native, timeline TBD
3. Google TPU v5e / Amazon Inferentia2 — existing cloud-native options

The Intel bridge chip revelation is a software ecosystem fragmentation red flag. If Nvidia can't use its own interconnects with the LPU, your serving frameworks (vLLM, TensorRT-LLM, Triton) will need adaptation for heterogeneous backends. Start abstracting your inference layer now — use framework-agnostic model formats (ONNX, SafeTensors) and ensure deployment can target multiple backends without model surgery.

Critical gap: zero published performance benchmarks. No tokens/sec, no latency, no cost-per-token comparisons. This is a $20B deal with marketing-grade technical detail. Any performance projection is speculative until independent benchmarks appear.

Planning Horizon

The confirmed GPU roadmap — current → Rubin → Feynman — with LPU fusion planned for the Feynman die tells you the endgame: a single chip that trains and infers optimally. That's 2028+ at the earliest. For infrastructure planning: optimize for heterogeneous compute now (2026-2028), but design your software abstractions for eventual convergence.

The Most-Cited Tool This Cycle

Karpathy's autoresearch appeared independently in 4 of today's intelligence sources — the highest convergence signal of the cycle. The tool runs an autonomous loop: an LLM agent generates hypotheses, modifies PyTorch training scripts, trains for exactly 5 minutes, evaluates validation loss, keeps improvements, discards failures, and repeats. The headline results across sources:

• 700 experiments over 2 days
• ~20 improvements survived (~3% keep rate)
• 18% hit rate at finding better configurations (reported as ~human researcher parity)
• 11% GPT-2 training speedup (2.02h → 1.80h)
• Runs on a single GPU, MIT-licensed, 3 files, no complex configs

Autoresearch isn't AI doing research — it's an LLM-powered hyperparameter search that found 11% speedup in 700 tries. That's still worth stealing for your training pipeline.

Reconciling the Numbers Across Sources

The 4 sources report slightly different framings that are worth reconciling. One source reports 18% success rate at finding better setups (matching human researcher hit rate). Another reports ~20 improvements out of 700 experiments — which is actually a ~3% keep rate. These aren't contradictory: the 18% likely measures per-hypothesis quality (the LLM proposes good directions 18% of the time), while the 3% measures end-to-end survival after training validation. Both are useful calibration points.

What This Is and Isn't

Multiple sources correctly identify the key constraint: your training task must produce meaningful validation signal in ≤5 minutes. This makes autoresearch immediately useful for:

• Small model architecture search
• Data augmentation strategy exploration
• Learning rate schedule and optimizer tuning
• Feature engineering experiments on fast-training models

It does not work for: your 7B fine-tuning job, large-scale pretraining runs, or any task where 5 minutes of training produces only noise in validation metrics.

One source raises an important methodological concern: Goodhart's Law applies. Optimizing 700 times against a single proxy metric risks overfitting to the evaluation harness. The greedy sequential selection also misses interaction effects — improvements that only work in combination. These are real limitations, but they're limitations of the specific implementation, not the pattern.

The Transferable Pattern

The broader idea — LLM-as-experimenter with hard resource constraints — is more interesting than the specific tool. The agent modifies architecture, optimizer, data augmentation, anything in the training script, not just hyperparameters. This is hypothesis-driven experimentation, not grid search with extra steps. Even if the LLM-generated hypotheses are mediocre, the workflow automation alone saves hours of manual config editing.

The comparison to production multi-agent patterns is instructive. One source notes that the top 5-10% of builders using AI tools are 3-5x more productive by orchestrating fleets of agents rather than single-tool workflows (Sequoia VC estimate, anecdotal). Autoresearch is a concrete, narrow instance of this fleet pattern: one agent proposing, one executing, one evaluating, running in a tight loop.