The Engineer

The Broken Assumption

Most teams price agentic workloads like chat: input tokens × per-token rate. For agents that math is wrong, and wrong in a measurable way. The dominant cost in a multi-step agent isn't compute or egress. It's KV cache residency, the GPU memory held between tool calls so you don't re-prefill the context.

Here's what actually happens. Each step re-sends the same long prefix: system prompt, tool schemas, prior observations. If the cache is warm, that prefix is free. If it evicted between steps because the provider's TTL expired, you pay full prefill again. Most hosted providers evict at ~5 minutes. DeepSeek's disk-based cache persists for hours.

On a twelve-step agent with tool outputs, KV cache residency is the line item. On short chats it's rounding error. The agentic cost model most teams use is broken in a specific way.

The Numbers

A DeepSeek billing screenshot one operator posted this week shows $1,050 in actual spend against $3,351 in cache savings. That is a 3.2x gap between per-token sticker price and effective cost on a single account. DeepSeek V4 Pro's hybrid CSA/HCA attention also reports a 10% KV cache size reduction at 1M context and roughly 4x lower inference FLOPs at long context. Cheaper per GB and longer-lived compounds over the hours of a real agent session.

MoE Makes It Worse

The second failure mode is Mixture-of-Experts cost modeling. A 49B-active MoE inside a 1.6T total parameter model has two cost drivers that don't collapse. Active parameters set the matmul bill per token. Total parameters set the minimum memory footprint and shard size. Most TCO spreadsheets fold both into one FLOPs figure, which mispredicts both latency and unit economics. Two models that looked competitive on per-token pricing were not competitive once cache residency was charged honestly.

The Fix

The corrected model has three components:

1. KV cache: price by GB-hour at the accelerator memory rate
2. Active parameters: price by FLOPs per token (not total parameter count)
3. Shard footprint: price by minimum deployable instance forced by total parameters

Then run an actual agent trace through that model, not a synthetic benchmark. The ranking of which models are cheapest for agent work will change. Independent writeups keep landing in the same place: on long-horizon agent traces, cache residency dominates per-token price.

The Harness Multiplier

Token-efficient harness design amplifies these savings. Hugging Face is shipping concrete patterns: agents.md files that front-load context an agent would otherwise scrape from docs, and token-efficient API responses that strip verbose JSON envelopes. Every token saved in a response is a token that doesn't compete for cache space. Smallest stable prefix across agent steps wins, not lowest per-token rate.

The Convergence

Three open-weight MoE models shipped in one week. All landed within 6–8 points of GPT-5.5 on the Artificial Analysis Intelligence Index:

The remaining gap is concentrated in HLE, TerminalBench Hard, CritPt, and Omniscience. Hard reasoning frontiers. On coding, tool use, and multi-step planning — the actual agent workload — these models are functionally at parity. The 6-point delta that justified frontier API pricing is smaller than the delta between a decent harness and a sloppy one.

Specialized Models Are Dead Weight

OpenAI killed standalone Codex this week and folded it into GPT-5.5. The message is not subtle: general-purpose models have reached coding-task sufficiency. Mistral shipped the same signal by collapsing three models into one flagship. Alibaba's Qwen3.6-27B outperforms its 400B+ predecessor on coding benchmarks at 15x smaller and single-GPU deployable. Routing logic that selects a specialized model per task is becoming dead code.

Stop building routing logic that selects specialized models per task. Build a clean abstraction layer that lets you swap the underlying model without touching business logic. You'll need it again in 6 months.

Local Inference Became Practical

PFlash speculative prefill hits 10x over llama.cpp at 128K context on an RTX 3090, using a Qwen3-0.6B drafter. Qwen 3.6 35B-A3B runs on an AMD 7700 XT at 128K with flash attention. The floor for self-hosted, cache-optimized agent infrastructure dropped this week.

There is a catch that several sources flagged independently. Open-weight models benchmark well and underperform in agent workflows. The reason is mechanical: LangChain, OpenCode, and similar harnesses are tuned to proprietary API conventions — tool-call format, system prompt layout, retry behavior. Swap in a local model and the scaffolding is wrong. The fix is not a better model. The fix is a model-specific harness. Log tool-call traces from the API run and the local run, diff them, build the adapter.

Platform Design Shifts

Hugging Face is redesigning for agents as the primary consumer by end of 2026. Three patterns worth adopting now: agents.md at repo roots for machine-readable project context, token-efficient API responses with small payloads and short stable field names, and headless-first interfaces where the CLI is canonical and the UI is a view. This is not vision. It is shipping code from a team with 15M users and 200 engineers.

The Pattern

The new agent security primitive worth naming this week is the Planner/Executor Split. Two LLM instances. A hard privilege boundary between them. The planner sees the tools and the plan, never untrusted input. The executor sees untrusted content, never the tools. Prompt injection lands on the component that has no authority to act.

This isn't a prompt engineering trick. It's privilege separation ported to LLMs, the same mechanism Unix uses to keep user processes out of root. Gmail already runs stacked defenses on this pattern in production.

Skip the planner/executor split and every tool call is one clever paragraph away from exfiltration.

MCP vs Skills: A Security Boundary Decision

MCP-versus-Skills gets pitched as a bake-off. Read the capability surface and it's a security boundary decision. The asymmetry that matters:

A Skill that ingests untrusted content and shells out is a prompt injection to RCE chain. Not theoretical. The split I'd ship: MCP for anything touching production. Skills for dev-facing agents in CI runners, ephemeral containers, and workstations. Nowhere else.

The Defense Stack

The complete taxonomy has two layers. One makes injection harder. The other caps what happens when it lands.

Model-level (probabilistic — makes injection harder)

• Spotlighting: wrap untrusted text in control tags (<UNTRUSTED>) and tell the system prompt to treat tagged content as data only
• Instruction Hierarchy: fine-tune the model to rank system prompts above user messages above third-party content

System-level (deterministic — caps blast radius)

• Planner/Executor Split: hard privilege boundary, 2x inference cost
• Least-Privilege MCP: minimum operation set per server, typed schemas that reject unexpected params
• Human-in-the-Loop: checkpoint gates on irreversible actions

The Planner/Executor Split is not free. 2x inference, coordination latency, and a handoff protocol you have to design and maintain. For any agent that reads external content and takes write actions, it's the only pattern that stops a single injection from doing damage. Pay the tax.