The Engineer

The Quantization Breakthrough

RotorQuant applies Clifford Algebra rotors to vector quantization, reducing computational complexity from 16,384 FMAs to ~100 for d=128. This isn't an incremental optimization — it's a fundamentally different algorithm. The cosine similarity trade-off is negligible: 0.990 vs TurboQuant's 0.991. Fused CUDA and Metal shader implementations are already shipping, outperforming cuBLAS matmul on RTX PRO 4000 and Apple M4.

RotorQuant achieves 10-19x speedup over Google's TurboQuant with 44x fewer parameters. This has shipped, not just published.

The 3-Line KV Dequant Fix

A complementary optimization exploits attention sparsity in KV dequantization: skip 90% of dequant work for tokens with negligible attention weights. The result is +22.8% decode speed at 32K context on M5 Max, and a jump from 0.45x to 0.73x relative to q8_0 KV cache on M2 Pro. This is the kind of fix that makes you ask why we weren't doing this already. Most inference deployments running quantized KV caches at long contexts are leaving 20%+ performance on the floor.

Why This Matters More Than Usual: H100s Are Appreciating

H100 rental prices have reversed their depreciation curve since December 2025 and are now worth more than at launch over 3 years ago. The driver is structural: reasoning models and agent workloads demand longer inference chains, larger KV caches, and more concurrent sessions. The AA-AgentPerf benchmark now measures throughput as 'concurrent users per accelerator per kW per dollar per rack' at 100K+ sequence lengths — that's a capacity planning metric, not a research number.

Connect these data points: GPUs cost more, not less. Agent workloads are getting heavier, not lighter. Microsoft is down 34% because investors doubt the ROI on AI infrastructure. Every token saved via RotorQuant, every dequant skipped via attention sparsity, directly translates into serving more sessions per GPU-dollar. These aren't micro-optimizations — they're the difference between a viable inference business and an unprofitable one.

The Qwen Deployment Signal

TurboQuant already enables Qwen 3.5-9B on a MacBook Air (M4, 16GB) with 20K token context. A vLLM fork targets Qwen3.5-35B AWQ with 1M context and 4M KV cache. RotorQuant's improvements on top of these baselines push the envelope further. If you've been waiting to bring serious models to edge or local hardware, the math just changed.

The Sora Precedent

OpenAI didn't deprecate Sora with a 12-month migration window. They killed it. In doing so, they destroyed a planned $1 billion, three-year partnership with Disney. If Disney — one of the most powerful enterprise customers on earth — can get burned by an AI platform dependency, your production integration is not safe either. This is the strongest real-world argument yet for the adapter pattern around AI providers.

Your AI gateway shouldn't just handle retries and rate limits — it should swap providers for any capability without touching application code. Disney just learned the cost of the alternative.

Anthropic's Capacity Crunch

Anthropic is actively throttling Claude due to demand surges while simultaneously licensing models to Yahoo Scout for 250 million users. If your production systems call Claude APIs, you're now sharing capacity with a quarter-billion-user platform. This is the noisy neighbor problem at the API layer. Meanwhile, Anthropic's infrastructure showed visible strain — 529 errors during the Capybara leak period. The rumored Capybara model (above Opus, potentially ~10T parameters) would have punishing per-token serving costs, further straining capacity.

The Budget Squeeze Compounds the Risk

Microsoft is down 34% since October — its worst quarter since 2008 — specifically because shareholders are revolting against AI capex without clear returns. This changes the internal politics of every AI project. For three years, 'AI' was a magic word that unlocked budget. That era is ending. CFOs are reading about Microsoft's bloodbath and recalibrating.

The rate environment is tightening simultaneously: Fed rate expectations flipped from 90% probability of cuts to 52% probability of hikes within a single month. Startup runway — and the AI vendors built on it — just got more fragile.

The Engineering Response

Three layers of defense:

1. Provider abstraction: Build an LLM gateway that can swap providers per capability. Not just chat — embeddings, vision, code generation, transcription. Each capability should have a primary, secondary, and local fallback.
2. Circuit breakers and fallback: Treat LLM APIs like any unreliable external dependency. Implement request prioritization, graceful degradation, and tested failover paths.
3. ROI documentation: Every AI infrastructure proposal now needs cost-per-inference, revenue attribution, and break-even timeline. The era of 'AI' as a blank check is over.

The Shift in Framing

Jack Dorsey told a room of JPMorgan investors that using Goose — Block's open-source autonomous coding agent — for a few hours each morning convinced him he could cut Block's engineering headcount by ~50%. Databricks CEO Ali Ghodsi described the same pattern: personally using coding agents, then using that experience to pressure his engineering team. This is not the developer tools pitch you've been hearing.

These two framings will collide in your organization within the next quarter. When your VP of Engineering gets asked in a board meeting 'Jack Dorsey says he can halve Block with AI — what's our plan?' you want the answer ready.

The Data Tells a More Nuanced Story

Research shows AI tools increase output volume by 42% without improving individual success rates. LLM code generation produces vulnerable code 30% of the time in controlled tests. More PRs, more generated code, same (or lower) average quality. The gap between CEO perception and engineering reality is wide and growing.

The engineers who thrive won't be those who ignore AI tools or those who panic — they'll be the ones who demonstrably use AI to do work that wasn't possible before.

Goose Is Real and Open-Source

Block has open-sourced Goose (github.com/block/goose) — an autonomous coding agent that operates across your entire dev environment, not just autocomplete. The fact that a non-engineer CEO finds it usable for hours daily suggests a fundamentally different interaction model than Copilot or Cursor. The gap between 'suggests code completions' and 'autonomously executes multi-step development tasks' is where the real disruption lives.

Your Response Strategy

Ground your narrative in data, not defensiveness:

• Measure and publish how AI tools improve your team's code quality, reduce incident rates, and enable harder problems — not just velocity
• Adopt autonomous agents (Goose, Claude Code, Codex) visibly, so leadership sees your team leading adoption rather than resisting
• Quantify the quality gap: the 30% vulnerability rate in AI-generated code means more output without better review creates technical debt and security risk — frame AI savings against the cost of the bugs it introduces