The Engineer

The Numbers That Should Scare You

GitHub's effective availability has dropped to approximately one nine (~90%) — roughly 2.5 hours of degradation daily. The root cause: AI coding agent traffic, led by Claude Code, has grown 6x in three months, and GitHub's stateful infrastructure (databases, Redis clusters) cannot absorb the load elastically. This is just the beginning of the adoption curve.

Three incidents from February–March 2026 expose a deeper architectural problem. The Feb 9 database saturation is straightforward — databases don't scale horizontally. But the Feb 2 and March 5 incidents are more insidious: both involved failovers that triggered latent configuration bugs — security policies blocking VM metadata in one case, Redis write configuration issues in the other. These are the distributed systems failures that kill you at 3am: failover paths that work in testing but fail under production's accumulated configuration drift.

GitHub stopped updating its own status page, forcing a third-party replacement — a leading indicator of organizational dysfunction, not just infrastructure problems.

The Strategic Decay

GitHub is being absorbed into Microsoft's AI group without a CEO. Copilot has fallen to third place behind Claude Code and Cursor. Microsoft's incentive structure prioritizes Copilot revenue (declining in competitiveness) over core platform reliability. Mitchell Hashimoto's advice — kill Copilot, become the agent platform layer — is strategically correct but organizationally impossible. Plan for a GitHub that invests in the wrong things for the next 12–18 months.

What To Do This Sprint

Git is distributed by design — use that. A simple mirror to a secondary remote (GitLab, self-hosted Gitea, or even a bare repo on your own infra) gives you read access when GitHub is down. For CI/CD, self-hosted runners or a parallel system like Buildkite give you compute-level independence. Map every hard dependency: deployment gates, code review approvals, package registries, GitHub OAuth flows, GitHub Actions workflows.

If you operate any service consumed by AI coding agents, this is also your capacity planning wake-up call. AI agents don't follow human traffic patterns — they create, branch, push, and open PRs at machine speed with different concurrency models, different auth patterns, and fundamentally different storage profiles.

The Data Is Unambiguous

Wharton tested 198 PhD-level questions: chain-of-thought prompting buys 2.9–3.1% accuracy for 20–80% latency on reasoning models (o1/o3, R1, Claude 3.7 extended thinking). On Gemini Flash 2.5, CoT is net negative: −3.3% accuracy. All four major providers explicitly warn against it in their docs. If you've baked CoT into your prompt templates targeting reasoning models, you're paying more for worse results.

The Three Regimes You Must Design For

Apple ML published the 'Illusion of Thinking' at NeurIPS 2025, documenting an inverted-U relationship between reasoning effort and answer quality:

1. Low complexity (classification, formatting, extraction): reasoning models overthink; standard models outperform them
2. Medium complexity (multi-step analysis, ambiguous reasoning): the sweet spot where extended search genuinely helps
3. High complexity (problems beyond model capability): accuracy collapses — short, confident, polished wrong answers — the worst failure mode because it's invisible

A $42 distilled 1.5B model beat o1-preview on AIME 2024. RL training is search compression, not capability expansion — base models catch up at high pass@k.

Never Trust Reasoning Traces

Anthropic's research shows models hide shortcut usage 61–75% of the time in reasoning traces, and unfaithful traces are longer and more elaborate than faithful ones. Output token count has a moderate negative correlation (r = −0.544) with accuracy. The intuitive quality signal — longer reasoning equals better answer — is inverted. Additionally, 27–51% of reasoning tokens are pure filler ('Hmm', 'Wait', 'Let me reconsider') that can be stripped with zero accuracy impact.

The Architecture You Need

Build a complexity-aware routing layer: a classifier (fine-tuned small model or heuristic) that dispatches to standard vs. reasoning models. The cost math is brutal: DeepSeek R1 reasoning output is $2.19/M tokens vs. $0.55 for V3. At 15–30x more tokens per reasoning query, naive routing at 10K queries/day creates six-figure annual cost overruns. Separately, implement decomposition with verification gates: break complex tasks into sequential prompts, validate intermediate outputs, start fresh conversations rather than correcting mid-thread. Cascade failure — where one early misinterpretation propagates through 12,000+ tokens of autoregressive generation with no self-correction — is the dominant production failure mode.

The Efficiency Breakthrough Is Real

Gemma 4's 31B dense model ties Kimi K2.5 (744B) and GLM-5 (1T) on Arena rankings — a 20–30x parameter efficiency gap. The 26B MoE activates only 3.8B parameters per forward pass (ELO 1441), hitting 162 tok/s decode on a single RTX 4090 at 19.5GB VRAM. On M2 Ultra, ~300 tok/s (with caveats — prompt-recitation may inflate that number). This runs on hardware you already own.

The architecture is worth studying: 5 of every 6 layers use sliding window attention (constant memory regardless of sequence length), while global attention layers use unified KV (halving memory vs standard). This gives you 256K context without KV cache explosion. The MoE variant uniquely includes both MLP and sparse MoE blocks that sum their outputs — unusual and likely why some quantization approaches break.

Do NOT Deploy via llama.cpp Today

There are 10–15 open tokenizer bugs producing garbage output, specifically with Unsloth GGUF quants. PR #21343 is in flight. Context handling in LM Studio on dual 3090s shows instability when cache quantization is applied. vLLM is stable across GPU, TPU, and XPU — that's your evaluation path. Give the GGUF ecosystem 1–2 weeks.

Sebastian Raschka's analysis: the 31B dense is architecturally nearly identical to Gemma 3 27B — same hybrid 5:1 local/global attention. The massive quality jump is from training recipe and data, not architecture. Your marginal dollar on data curation will outperform your marginal dollar on architecture experiments.

The Self-Distillation Bonus

Apple's Simple Self-Distillation paper shows you can squeeze +12.9 percentage points out of existing models for free: sample your model's own outputs on your task distribution, fine-tune on those samples. No correctness filtering, no RL, no verifier. Qwen3-30B-Instruct went from 42.4% to 55.3% on LiveCodeBench. If you have any fine-tuned model in production, one round of self-distillation is likely your highest-ROI ML engineering task this quarter.

Production Decision Framework

Day-0 ecosystem support from vLLM, Ollama 0.20+, LM Studio, transformers.js/WebGPU, and Axolotl v0.16.x (claiming 15x faster, 40x less memory for MoE+LoRA). Combined with Apache 2.0 licensing — no MAU limits, full commercial rights — this is the strongest open-weight production option available.