The Scientist

The Convergence

Two independent intelligence streams this week point to the same conclusion: coding agents are production-ready in capability but production-dangerous without scaffolding. Anthropic's Opus 4.6 benchmarks reveal a steep reliability decay — 80% accuracy at 1-hour tasks, dropping to 50% at 14.5 hours — while OpenAI, Stripe, and Anthropic are independently converging on a discipline called harness engineering to solve exactly this problem.

The throughput numbers are seductive: 3 engineers built a million-line product in 5 months at OpenAI with zero hand-written code. Stripe's internal agents produce 1,000+ merged PRs per week. A solo developer made 6,600+ commits in a single month running 5-10 agents simultaneously. But the compound error math is brutal — chain 5 sub-tasks at 80% reliability each, and your end-to-end success rate drops to ~33%.

The bottleneck for coding agents is never the agent's ability to write code — it's the lack of structure, tools, and feedback mechanisms surrounding it. This is the MLOps lesson all over again: the model isn't the bottleneck, the system around it is.

The Emerging Playbook

Here's what the convergent practices look like, translated for ML teams:

The Verification Gap — Worse for ML

Anthropic discovered that agents mark features as complete without proper end-to-end testing. For application code, this produces visible bugs. For data science code, the consequences are far more insidious: a subtle feature transformation error that passes unit tests but introduces distribution shift is invisible until it degrades production predictions. No harness pattern yet addresses statistical correctness verification — this is the open problem your team should be thinking about.

Where This Doesn't Work Yet

• Legacy ML codebases — your existing Airflow DAGs, undocumented Spark jobs, and notebook-to-production scripts are the brownfield problem that remains unsolved
• Review bandwidth — human reviewers become the bottleneck at 3-4 parallel agent sessions, and reviewing agent-generated statistical code requires more domain expertise per line than CRUD endpoints
• Code entropy — agent-generated code accumulates cruft differently than human-written code, manifesting as redundant preprocessing steps and unnecessarily complex transformations

Three Signals, One Threat Model

Three independent sources this week converge on a single uncomfortable truth: the agentic AI stack is scaling faster than its security. Cisco's SVP of AI confirms attackers are "already probing gaps" in MCP and agent-to-agent protocols. A Cambridge-led study found only 4 of 30 top AI agents (13%) have published formal safety evaluations — with browser agents missing 64% of safety disclosures despite being the most autonomous category. And Amazon reports that 600+ Fortinet firewalls were breached across 55 countries in weeks by a small group using commercial AI tools — attack vectors were mundane (weak passwords, exposed ports), but AI enabled scanning and exploitation at unprecedented scale.

If you're deploying agents without checkpoints and your own evaluation harness, you're the one running the experiment — and attackers are already probing the same infrastructure your agents connect to.

The Dual Threat

Cisco's framework identifies two distinct threat vectors that require different mitigations:

1. Protecting the enterprise from agents — rogue or compromised agents acting against organizational interests (data exfiltration, unauthorized privilege escalation, production environment modification)
2. Protecting agents from the world — adversarial inputs, prompt injection, and data poisoning targeting agent decision-making

The proposed security layers are sound in principle:

• Zero-trust identity — every agent treated as an untrusted actor requiring continuous verification
• Control over agent protocols and tool registries — governing what agents can access and execute
• Continuous behavioral monitoring — detecting anomalous agent actions in real-time

No implementation specifics, detection rates, or false positive management data were disclosed by any source.

MCP: Standard and Target Simultaneously

MCP is coalescing as the de facto agent-to-tool integration layer — Stripe exposes 400+ internal tools via their Toolshed MCP implementation, Claude Code uses it as the reference integration point for Gmail/Slack/Notion/Calendar, and third-party tools like runCLAUDErun are building on it. But this standardization creates a monoculture risk: a vulnerability in MCP's protocol layer would affect every agent ecosystem built on it.

For ML teams specifically, MCP is how you'd expose your feature store CLI, experiment tracker, model registry, and data catalog to agents. Every one of those tools contains sensitive data about your models, training data, and production infrastructure. The question isn't whether to adopt MCP — it's whether your security posture is ready for it.

Your Specific Threat Surface

• Can an attacker impersonate one of your agents to another via MCP?
• Can a compromised agent escalate its own privileges through tool-call chains?
• Are you logging all agent actions with enough granularity to detect behavioral anomalies?
• Do you have hard human-in-the-loop gates before any agent can modify production environments or trigger irreversible operations?

OpenAI Can't Forecast Its Own Costs

OpenAI's leaked financials reveal a cost structure that should recalibrate every ML team's inference budget planning. Model serving costs quadrupled in 2025, compressing gross margins to 33% — a 13-percentage-point miss against their own internal forecast of 46%. Revenue hit $13.1B (slightly above forecast), but the cost side tells the real story: the company now projects $111 billion in cumulative cash burn through 2030, more than double its previous estimate.

The path to profitability rests on a single assumption: training costs drop ~$28B in 2030, fully offsetting inference growth. This requires either post-Blackwell hardware delivering dramatic FLOPS/dollar improvements or algorithmic breakthroughs reducing compute by an order of magnitude. Neither is guaranteed, and both are outside OpenAI's direct control.

OpenAI can't forecast its own inference costs one year out (33% vs 46% gross margin), so treat anyone's 5-year AI cost projections as fiction and build your pipelines for cost uncertainty, not cost decline.

The Physical Bottleneck

The cost pressure has a physical dimension. New AI campuses are designed for 1 gigawatt of power or more — over 10x traditional data centers. The executives who can build at this scale are commanding $10M+ compensation packages, and lenders are inserting key-man clauses that can pull project financing if a specific executive leaves mid-build. Nvidia's expected 67% YoY revenue growth to $65.7B confirms GPU demand still outstrips supply.

For your planning: if the binding constraint on AI scaling is now physical infrastructure (power, cooling, construction) rather than chip fabrication, GPU availability growth may plateau or become lumpy. Wednesday's Nvidia earnings call guidance matters more than backward-looking numbers — listen for Blackwell ramp timeline and supply constraint commentary.

What This Means for Your Stack

If you're consuming OpenAI APIs or any hosted LLM, a 33% gross margin is not sustainable for a company burning $111B. Either prices go up, rate limits tighten, or cheaper models get pushed more aggressively. The engineering response:

• Build model-routing logic that dynamically shifts between providers and model sizes based on cost/quality tradeoffs
• Benchmark smaller open-source models (Llama 3, Mistral) against your specific use cases — the quality gap may be smaller than you think for many production tasks
• Invest in efficiency techniques (distillation, quantization, MoE) as a hedge against both cost increases and capacity constraints
• Stress-test your 2026-2027 inference budget against scenarios of 2x-3x price changes in either direction