The Scientist

The Pricing Earthquake

Eight independent sources covered GPT-5.4 mini and nano this week, and the consensus is clear: the bottom of the inference stack just got dramatically cheaper. Nano lands at $0.20/M input tokens and $1.25/M output — API-only, purpose-built for classification, extraction, and ranking. Mini arrives at $0.75/M input, $4.50/M output, with a 400K context window, and scores 54.38% on SWE-bench Pro (up from GPT-5 mini's 45.69%, a 19% relative improvement).

At nano pricing, you can run 5 million classifications for $1. That shifts the break-even for maintaining custom fine-tuned BERT or logistic regression classifiers — when you factor in training compute, labeling, infrastructure, and retraining cadence, the total cost of ownership for self-hosted models is suddenly harder to justify below millions of daily inferences.

Where Sources Agree — and Disagree

All eight sources agree on the pricing and strategic positioning. The divergence is on whether these models are actually good enough. One source reports mini scored "relatively low" on BullshitBench — a benchmark testing resistance to false premises and jargon. Another flags a 24.5% Pass@1 on APEX-Agents for agentic tasks. A third source claims OpenAI hiked prices 4x versus predecessors, directly contradicting the "cheaper inference" narrative — the models are cheaper than GPT-5.4 full, but may be more expensive than the GPT-5 mini/nano they replace.

Zero sources published multi-benchmark quality comparisons, latency methodology, or ablation studies. "Outperforms predecessors" without evaluation harnesses is marketing, not science.

The Three-Tier Routing Architecture

The practical architecture is now obvious: nano for high-volume extraction and classification, mini for coding/reasoning/tool-use, full GPT-5.4 for frontier tasks. Even a rule-based router (task type → model tier) can cut inference costs 30-50%. A learned router that classifies query complexity is the next step. Multiple sources independently converge on this pattern — it's the new default for any pipeline running more than 10K daily inference calls.

But there's a trap: small models degrade unpredictably on distribution tails. Nano may handle 95% of your classification traffic beautifully and silently fail on the 5% that matters most. Set up automated quality monitoring with distribution shift detection on nano outputs from day one. Stratify your evaluation by input difficulty — aggregate accuracy will mask the failures that cost you.

The Fine-Tuning Calculus Shifts Again

Every API pricing drop changes the build-vs-buy math. If your fine-tuned BERT requires GPU hosting at $0.50-2/hr, the break-even volume against nano at $0.20/M is surprisingly high. Run the numbers for your specific volume, latency SLA, and accuracy requirements before your next model retraining cycle. The comparison isn't nano's accuracy vs. your model's accuracy — it's nano's accuracy × $0.20/M vs. your model's accuracy × (hosting + training + labeling + maintenance).

The Architecture OpenAI Published

OpenAI released a rare production architecture teardown of Codex, and the headline for ML practitioners is counterintuitive: the hardest problems had almost nothing to do with the AI model itself. The codex-1 model (a fine-tuned o3) is described as one component in a much larger system. The real engineering lives in the orchestration layer — prompt assembly from 5+ sources, context management, multi-surface deployment, and a protocol stack they had to rebuild from scratch after MCP failed.

The cost math is stark: each turn in a multi-turn agent resends full conversation history — quadratic data transfer by design. For a 20-turn conversation adding ~2K tokens per turn, you're transmitting roughly 420K tokens total even though net new content is only 40K. With perfect prefix caching, you compute on just the 40K. Without it, you compute on all 420K. That's a 10x cost difference hinging entirely on cache integrity.

The Bug That Should Terrify You

When OpenAI added MCP tool support to Codex, a bug where tools weren't listed in consistent order between requests was enough to destroy prompt cache hits entirely. This is the kind of silent cost multiplier that passes every functional test — your agent produces correct outputs, your latency metrics look normal — but your inference bill explodes because every request triggers full recomputation.

A non-deterministic tool ordering bug can silently 10x your inference costs with zero functional test failures. Cache hit rate is as critical as model accuracy for production agents.

Context Engineering Converges From Five Directions

This week, five independent sources converged on the same thesis: context engineering outranks model selection for agent reliability and cost. The patterns are complementary:

• Cursor's RL-trained self-summarization: Instead of prompting for summaries, they trained Composer via RL to compress earlier context, cutting compaction error by 50% and extending effective working memory. If you have any pipeline that truncates or summarizes context, this technique generalizes immediately.
• Anthropic's folder-based skill packages: Internal data from hundreds of Claude Code skills shows structured folders (scripts + reference code + templates + config) dramatically outperform single markdown prompts, with nine distinct skill archetypes identified.
• Context pollution from extensions: A practitioner at a robotics startup warned that OpenClaw users add extensions without considering how they inflate context — the same signal-to-noise problem as feature engineering, now applied to agent prompts.
• Autoresearch findings: Environment design and validation gates outperform model choice for preventing agent drift; GPU costs from rejected proposals dominate total compute spend.

MCP's Documented Limitations

OpenAI tried MCP for VS Code integration and it failed. Rich interaction patterns — streaming progress, mid-task user approval, structured code diffs — didn't map to MCP's current capabilities. They built a custom bidirectional JSON-RPC protocol (App Server) from scratch. If your agent pipeline requires any of these patterns, plan for a custom protocol layer alongside MCP, not instead of it.

Two Models, One Week

In a single week, two open-source OCR models arrived that challenge every commercial document extraction pipeline:

GLM-OCR at 0.9 billion parameters achieving #1 on OmniDocBench is a remarkable parameter efficiency result — a 100x+ reduction compared to larger competitors. Chandra OCR 2 halving its parameter count from 9B to 4B while achieving SOTA on olmOCR shows the efficiency frontier in document AI is collapsing rapidly.

What's Missing Before You Switch

Both models ship with significant methodology gaps. Chandra provides a single benchmark score (olmOCR) with no per-category breakdown — how much of that 85.9% comes from clean typed documents versus the hard cases (handwriting, complex tables, low-resource languages)? GLM-OCR provides no architecture details, no training data description, and no cross-benchmark validation. Neither model publishes a head-to-head comparison against Google Document AI, AWS Textract, or Azure Form Recognizer on shared benchmarks.

The 4-point gap between Qwen3.5-9B (77.0) and GPT-5.4 (81.0) on document AI benchmarks is a surprisingly tight race for a 9B open model — adding Chandra and GLM-OCR to the mix makes commercial OCR increasingly hard to justify on cost alone.

The Eval You Should Run This Week

Build a stratified test set from your actual documents: 100 clean typed docs, 100 complex tables, 50 handwritten samples, 50 multilingual docs. Measure field-level F1, not just page-level accuracy. Run Chandra OCR 2 via vLLM and GLM-OCR via Ollama against your current commercial provider. If either open-source model matches at even 90% of the accuracy, the cost savings from single-GPU inference with zero API costs likely justify the switch. Chandra's structured JSON output with bounding box coordinates maps directly to downstream validation and human review workflows.

The capability claims are broad — complex tables with merged cells, inline LaTeX, handwritten cursive, form reconstruction with checkboxes — but these are exactly the failure modes that vary wildly by domain. Trust your own eval, not the benchmark.