The Scientist

The Shift Nobody Priced In

Agentic workloads spend most of their tokens inside loops: retries, tool calls, multi-turn reasoning against stable system prompts. Repetitive prefixes are exactly what KV cache reuse exploits. DeepSeek V4 Pro ships a disk-backed KV cache that persists for hours, versus the roughly 5-minute TTL that is the industry standard. One shared production dashboard reported $3,351 in cache savings against $1,051 in API spend. That is a 3.2× effective discount, and it does not show up on a price-per-token comparison sheet.

This is not really a DeepSeek story. It is a serving-architecture story. V4 Pro's hybrid CSA/HCA attention compresses KV cache to 10% of standard size and reports ~4× lower long-context FLOPs. At the concurrency and context lengths a real coding agent harness produces, effective cost between models swings 2–4× depending on serving stack. That delta is larger than the quality gap between open and closed models on most coding tasks, which is the comparison most teams are actually running.

The Pricing Floor Has a Hidden Fee

Grok 4.3 sets the headline floor at $1.25/M input, $2.50/M output, 40–60% below Grok 4.2. xAI also introduced a $0.05 fee per safety-filter-blocked request. At a 2–3% filter rate on production prompts, that erodes token savings meaningfully at scale. Most cost dashboards won't catch it because they track tokens, not rejections.

A model that is cheap to serve is not the same as a model that is cheap to trust. The blended cost on a replay of real traffic is the number that decides the migration, not the sticker price.

Cross-Source Pattern

Four independent sources converge on the same read: inference economics, not benchmark scores, are the binding constraint on production model selection. Nebius paid $615M for Eigen AI specifically for inference optimization. Cursor's −23% gross margins show what a thin wrapper over expensive inference looks like on the P&L. Grok's own commentary notes the headline price cut may be subsidized by poor utilization and is unlikely to beat a well-cached DeepSeek workload.

What To Do

The immediate action is observability, not migration. Most agent runtimes emit tokens/sec and $/query but not cache-hit rate, prefix-reuse ratio, or effective $/1K tokens net of cache discounts. Those three metrics are now first-class cost variables. Stable system prompts and tool schemas are where the DeepSeek-style discount actually lives, which makes harness design a cost lever, not an infrastructure footnote.

The Evidence Harness-Dependence Is Load-Bearing

Cross-vendor benchmark comparisons are not just noisy. They are systematically confounded by the evaluation harness, and this week produced the receipts:

1. GPT-5.5 beats Opus 4.7 overall on the Intelligence Index but loses on PostTrainBench when evaluated inside the Claude Code harness. Same weights, different plumbing.
2. Grok 4.3 gained +321 Elo on GDPval-AA yet regressed on Vending-Bench 2 so badly that the agent reportedly preferred to 'sleep' rather than act. The same model on a different eval surface produces the opposite conclusion.
3. Grok 4.3 ranks #1 on CaseLaw and CorpFin but scores 11% on ProofBench. Within-model domain variance is wider than the between-vendor gap on any single domain.

Hugging Face's Clem Delangue frames this as the 'engine vs car' problem: closed APIs bundle routing, tool schemas, retry logic, and sometimes multiple models behind one URL. Benchmarking an open-weight model through a harness designed for Claude's tool-use schema produces degradation that looks like a model problem but is an instrumentation problem.

The Qwen Signal

Alibaba's Qwen3.6-27B reportedly beats a predecessor 15× its size on coding benchmarks. The thing this headline doesn't tell you is whether contamination was ablated (it wasn't disclosed), and coding benchmarks reward single-file completions rather than the multi-repo edits where production assistance actually fails. The cheapest experiment is an A/B on held-out pull requests, not a re-run of HumanEval.

Cross-harness comparisons correlate with production outcomes. They do not cause them, and they do not predict them well enough to justify a migration on their own.

The Fix

The eval harness itself needs to become a logged, versioned variable. Two changes make this concrete:

• Log harness config as an eval dimension. Prompt format, tool schema shape, retry policy, context budget all differ across model families and all affect measured quality. Re-test top open candidates (V4 Pro, Kimi K2.6, Qwen3.6-27B) with per-family tuned harnesses.
• Require vendors to publish harness methodology. Any procurement checklist that accepts a benchmark score without the harness definition, tool-call budget, and retry policy is accepting an unauditable claim.

The open-weight gap is now 5 points on the Intelligence Index (52–54 vs. 57–60), concentrated in HLE, CritPt, TerminalBench Hard, and hallucination-heavy Omniscience, rather than general coding or agentic tool use. On multi-turn agentic coding specifically, V4 Pro is called the first open-weight model that genuinely feels comparable to Codex or Claude Code. If internal evals confirm a gap of ≤4–5 points on task-completion metrics, a two-week shadow-traffic bake-off is warranted. If the gap is larger, re-baseline the evals first.

The Reference Architecture for Untrusted Content

The Planner/Executor Split is settling in as the default defense pattern for agents that touch untrusted content: emails, web pages, user uploads, RAG retrievals. The shape is simple. A planner LLM has tool access but never sees untrusted text. An executor LLM reads untrusted text but has no tools. Gmail is cited as a production reference. The honest cost is that it roughly doubles inference spend. That is the price that turns indirect prompt injection from catastrophic into recoverable.

The thing this doesn't solve is exfiltration through the planner's own tool calls. The eval you actually want is not "did the executor get fooled." It is "did a document change the planner's plan in a way the user didn't ask for." Build that eval before the architecture, or the architecture ships without a way to measure whether it works.

MCP vs SKILL.md: A Systems Decision, Not a Taste Decision

The framing that MCP and Skills are competing approaches is wrong. They are orthogonal primitives:

The security surface most teams miss: Skills execute arbitrary bash/python/curl in the agent's own environment with no sandboxing. This is remote code execution by design. Every SKILL.md file deserves the same review rigor as a Dockerfile. Without Firecracker, gVisor, or at minimum a seccomp-restricted subprocess, a compromised Skill is a privilege-escalation vector.

The cost asymmetry: an MCP server that should have been a SKILL.md costs you a container, a deploy pipeline, and an on-call rotation. A SKILL.md that should have been MCP costs you correctness on a slice of traffic. One is a line item. The other is a bug.

Stacking Defenses

OWASP ranks prompt injection as the #1 LLM threat, and the guidance is explicit: no single fix exists. The layered approach combines:

• Spotlighting: Wrap untrusted text in <UNTRUSTED> tags with a system-prompt rule. Hours of work, meaningful floor raise.
• Instruction Hierarchy: Fine-tune to rank system > user > third-party content.
• Planner/Executor Split: Architectural isolation for high-stakes paths.
• Least-Privilege Tools: Minimize blast radius per tool call.

Spotlighting is the cheap floor and worth taking today. Planner/Executor earns its doubled inference cost on any path that touches the open web, and less obviously elsewhere. Skill sandboxing is the one that teams defer and regret; a compromised SKILL.md in production is not a bug you recover from quickly.