The Engineer

The arithmetic: 10M output tokens/day of coding work at Opus 4.8 rates costs ~$250/day. The same work at Grok 4.5's token profile and pricing costs ~$36/day. Per-token price isn't the whole delta. It's a lower price times 4.2x fewer tokens per task. Architectures shelved in 2025 as too expensive are worth re-costing.

OpenAI's answer is structural. Sol/Terra/Luna tiering turns model selection into a routing problem. Terra delivers GPT-5.5-class performance at $2.50/$15, roughly half prior cost. Luna matches Grok's $1/$6 floor. Sol posts 80.0 on Artificial Analysis, a coding SOTA. One vendor spanning three price points, which practically demands a dispatcher.

How to Evaluate Without Getting Played

Databricks' Pareto methodology is the right frame: pass rate versus mean cost per task on multi-million-line codebases. Grok 4.5's 15,954 output tokens at $6/M is ~$0.096/task in output alone. Run that across thousands of daily CI-triggered generations and 'cheap' stops being cheap. Sebastian Raschka's companion finding says the same thing: top-scoring configurations are rarely optimal under a fixed budget. Put it on the wall. Caveat: the 4.2x figure comes from SWE-Bench Pro, which has known broken tasks. Treat vendor efficiency claims as hypotheses until they replicate on your traffic.

The Trap

Every number here is a land-grab price. Meta's entry is 'highly subsidized.' OpenAI and Anthropic shipped 5+ usage and SLA changes within 48 hours, so contract terms now move faster than release cycles. The major labs are newly public and burning cash monthly. The post-IPO playbook is predictable: prices rise and free tiers shrink; rate limits tighten too. The savings hold only if your architecture can follow the price floor next quarter. Couple to one provider today and you've traded a discount for lock-in exactly when lock-in is most expensive.

Build the routing layer so you can re-pick providers every quarter without touching application code. That is the move, not picking the cheapest model.

Most routing implementations evaluate every call independently. In a multi-step agent task, that switches models between steps. Every switch re-bills the accumulated context cold, because prompt caches don't transfer across models. Here's the counterintuitive part: a cache-hot Claude Sonnet can cost less than a cache-cold Haiku on the same context. A gateway chasing the cheapest model per call plausibly costs more than no routing at all.

The Reference Architecture

The design insight is where the decision lives in time. Stages 1-3 fire at the first call. Stage 4 pins everything after. That preserves cache warmth and amortizes router overhead across the task. Stage 3's standby fallback gives instant failover on rate limits without re-running the router.

Caveats

Plano (open-source, from Katanemo) implements this as an infrastructure proxy: YAML-configured, zero agent-code changes, auto-updating pricing catalogs. The claimed 2x cost reduction is plausible but unverified. No disclosed baseline, unknown task mix. Arch-Router's preference-data training likely skews toward models popular at collection time, so evaluate on your prompt distribution first. And declarative YAML will fight complex policies like quality-gated escalation or A/B routing.

A repricing war makes routing more valuable and naive routing more destructive. Routing belongs in a proxy layer, the way API gateways absorbed auth and rate-limiting. Embedding it in agent frameworks accrues tech debt that gets expensive precisely when providers reprice.

Route once per task, then pin — session affinity beats a smarter router model, because cache warmth is worth more than model choice.

Previous tool calling was structured output: the model emits JSON against a function schema, your code executes, results feed back. Programmatic tool calling inverts control — the model writes programs that coordinate tools, process intermediate results, and spawn parallel subagents. Your integration layer becomes a sandboxed execution environment for model-generated code. Mental model: an untrusted contractor with access to your Terraform configs — powerful, demanding blast-radius controls, audit logging, and rollback before you enable it.

Ultra mode makes the direction explicit: 4 coordinated agents lift Terminal-Bench from 88.8% to 91.9%. And ChatGPT Work operating 'across connected apps, websites, and files for hours' means these are persistent stateful processes, needing the operational rigor of any long-running service.

The Real Trade-Off

OpenAI is pulling orchestration down into model serving. If you run custom LangGraph or CrewAI pipelines for multi-step coding: adopt ultra mode and delete an infrastructure layer, or keep custom orchestration and retain observability into individual agent decisions. You can't have both. Decide which loss you can afford before the benchmark decides for you.

Context Is the Other Half

Muse Spark 1.1's self-compacting 1M context (JobBench 54.7 vs Opus 4.8's 48.4) attacks long sessions from the model side: it autonomously drops stale detail. The failure mode is quiet — drop an early-session constraint and you get silent corruption of the agent's world model, unverifiable by users. Explicit memory management (RAG, sliding window with summarization) at least shows what the system remembers. Benchmark averages and production reliability are different measurements. Either way, active context management is the differentiator for any agent persisting past one request-response cycle — design your strategy deliberately rather than inheriting the model's.

The model is no longer something you call — it's something that runs, and it needs the same operational controls as any service in your fleet.

Three independent developments converge on one thesis: agent pipelines need the trust engineering we spent a decade building for dependency management — on egress, ingress, and output.

Egress: Your Code Already Left

xAI trained Grok 4.5 on 'entire data from Cursor' — trillions of tokens of real developer interactions: completions, edits, context. If your engineers use Cursor on proprietary repos, your internal APIs, auth patterns, and business logic are potentially embedded in a model served via public API. Not a hypothetical leak — a documented training pipeline. The mitigation isn't banning assistants; it's tiered data boundaries: which repos allow cloud-connected assistants, which require local or self-hosted inference, and what telemetry controls apply per tier.

Ingress: Signed Tools Arrive

NVIDIA's agent skills repo (npx skills add nvidia/skills) is the quieter, more important release. Skills are portable instruction sets — not code libraries — running across Claude Code, Codex, Cursor, and Kiro on the open agentskills.io spec. The part worth copying: every skill is cryptographically signed against NVIDIA's trust anchor and scanned for prompt injection and supply-chain risk. The CUDA-X lock-in is real; the security pattern isn't tied to CUDA-X. If an LLM autonomously invokes tools in your stack: are your tool definitions signed? Scanned for injected instructions?

Output: Adversarial Validation Works

The dual-agent pattern — one agent generates claims, a second with different context tries to destroy them, false positives landing in a reviewable 'hallucination bin' — produced 126 real vulnerabilities in 5 months of autonomous security testing, arrived at independently by two practitioners. You don't fix hallucination by making one model more reliable; you fix it architecturally: separate generation from validation, make validation adversarial. The precision/recall knob is tunable — aggressive validation kills more hallucinations and some true positives, worth it wherever false positives burn expensive human review.

Treat agents like dependencies: verify what comes in, control what goes out, and never let a single model grade its own homework.