The Scientist

Two Thresholds Crossed Simultaneously

The open-weight ecosystem just delivered its most consequential week. Gemma 4 crossed ~2 million downloads in its first week — a 15x weekly download acceleration over Gemma 3 (129K/week) — with day-one support from vLLM, llama.cpp, Ollama, NVIDIA, SGLang, Docker, and Cloudflare. More importantly, Gemma 4 E2B runs at ~40 tokens/second on iPhone 17 Pro via MLX, making on-device inference production-viable for instruction-following and tool-calling workloads.

Red Hat published quantized Gemma 4 31B in NVFP4 and FP8-block formats with live instruction-following evals (reasoning and vision evals pending). The E4B variant — 8B total parameters, only 4B active via MoE — runs locally on modest hardware with native vision and MCP tool-calling. The 31B is available for free on Google AI Studio.

On-device inference at 40 tok/s means the question is no longer 'can we run locally?' — it's 'which workloads should we stop paying API fees for?'

Post-Training Pipeline Has 2-4x Headroom

Two independent results point to massive untapped efficiency in RL post-training:

FIPO extends reasoning traces from ~4K to 10K+ tokens — it doesn't just improve accuracy, it changes reasoning behavior by encouraging more thorough exploration. No ablation details or confidence intervals were provided, so treat exact AIME numbers as directional. The async RL result from OLMo 3 is arguably higher-ROI: a pure systems optimization (no modeling changes) that turns a 4-day RL run into a 1-day run. The critical missing detail is whether final model quality metrics hold.

Self-distillation offers a near-free post-training step

The "embarrassingly simple" technique: sample multiple completions from your model, filter by correctness, retrain on passing outputs. This reshapes token distributions toward high-quality generation paths without requiring a reward model, human preferences, or RL infrastructure. If the gains reproduce on your code models, this is the cheapest addition to any fine-tuning pipeline.

Context: The Broader Open-Weight Landscape

Gemma 4 isn't the only contender. Qwen3.6-Plus is reaching near-frontier performance on agentic benchmarks. Cursor achieved 1.84x faster MoE token generation on Blackwell GPUs via 'warp decode.' And the Muon optimizer is coming to consumer Blackwell cards via matmul+epilogue implementation, bringing training efficiency gains previously limited to datacenter hardware.

Meanwhile, Meta's Avocado model failed benchmarks 'across the board' before release, and Meta is pivoting to a hybrid open/closed strategy. Don't assume next-gen Llama models will be freely available or competitive.

Coordinated Data Poisoning Targets Your Distillation Bottleneck

Chinese labeling workers are deploying coordinated anti-distillation poisoning tools that produce surface-plausible but deliberately corrupted training data. This is not random noise injection — it's coordinated adversarial corruption using shared tooling, specifically designed to be undetectable in standard inter-annotator agreement checks. The labels look correct on inspection but carry subtle corruptions that degrade model performance in ways that won't surface in standard QA audits.

The attack targets the distillation bottleneck — the cheapest path to replicating frontier model capabilities. From an ML perspective, this is a distribution-level attack, not an instance-level one. You won't catch it by spot-checking 5% of labels. The corruption operates in the statistical relationships between features and labels — exactly where your model learns.

Standard data quality pipelines — duplicate detection, label distribution checks, annotator agreement metrics — will pass these batches as clean. The workers understand the ML pipeline deeply enough to target the training signal rather than the labels themselves.

Caveat: these claims come from newsletter reporting without peer-reviewed verification. However, the coordination signal is corroborated by a separate development.

The Unprecedented Lab Coalition

OpenAI, Anthropic, and Google formed an alliance through the Frontier Model Forum to share adversarial distillation countermeasure data. These companies compete ferociously on everything — the fact they're cooperating implies the distillation technique is:

• Effective enough to erode competitive advantage
• Scalable across model families, not just specific architectures
• Attributable — they can detect when it's happening via query pattern analysis

No specific defenses were disclosed. Watch for papers from these labs in the next 2-3 months on output perturbation, differential privacy on logits, query fingerprinting, or watermarking.

Agent Trace Data: The New Strategic Dataset

A convergent signal from multiple sources: agent trajectory data is becoming the new pre-training data. HF CEO Clem Delangue is calling for crowdsourced agent trace collection. Baseten advocates learning from production traces. The pi-share-hf tool launched specifically for sharing coding-agent sessions with PII defenses built in.

If you're running agents in production, start capturing structured traces with anonymization now. In 6 months, teams with large trace datasets will have a compounding advantage in agent fine-tuning. The GrandCode paper introduced Agentic GRPO — an RL method purpose-built for multi-stage agent rollouts with late rewards — that beats every human participant in live Codeforces contests. Training methods like this need trajectory data as fuel.

If You Serve Models via API

The lab coalition signals model extraction is now a production threat. Audit your API surface: which endpoints return logits, probabilities, or embeddings? These are highest-risk for distillation attacks. Implement query logging and anomaly detection — systematic querying for distillation has distinctive patterns. Consider output perturbation: adding calibrated noise to logits degrades distillation quality with minimal impact on legitimate users.

The Distribution Shift Is Breaking Assumptions

GitHub commits are on track for 14 billion in 2026, up from roughly 1 billion in 2025. Claude Code public commits grew 25x in 6 months (100K/week → 2.5M/week). Agent-submitted PRs surged from 4M to 17M between September 2025 and March 2026. GitHub's COO confirmed every month since January 2026 has set peak usage records, while the platform simultaneously migrates from its own servers to Azure.

Notice the asymmetry: commits grew 14x but PRs only 4.25x. Agents make many more commits per PR than humans — consistent with iterative, self-review workflows. This fundamentally changes the statistical properties of GitHub data at the commit level.

This isn't a growth story — it's a distribution shift that breaks assumptions baked into every system that depends on GitHub.

Your CI/CD Is Exposed

GitHub's API rate limits are already failing agent-heavy users. OpenClaw founder Peter Steinberger reported hitting limits repeatedly, noting the platform "hasn't been designed with agents in mind." If your training pipelines, model CI/CD, or experiment tracking trigger on GitHub events, you're exposed to increasing reliability risk. GitHub availability has dropped to ~90% under the load.

The monetization problem amplifies the risk: GitHub's flat per-person pricing doesn't capture agent traffic. Claude Code and Codex push millions of commits at zero usage fees. Infrastructure costs scale with traffic (superlinear) while revenue scales with seats (linear). Expect pricing model changes targeting API/agent usage — and OpenAI is considering building its own GitHub alternative.

Your Training Data Quality Is Eroding

If you train code models or use GitHub-sourced code in any capacity, the signal-to-noise ratio of public repos is declining fast. With 2.5M Claude Code commits per week to public repos alone, plus Meta's 'tokenmaxxing' culture (60 trillion tokens/month, 85K ranked employees competing on volume not quality), any post-2025 GitHub scrape needs aggressive quality filtering.

Consider commit provenance (human or agent?), commit message patterns, and code churn rate as features for data quality classifiers. A single PR may now contain dozens of agent-generated commits that look structurally different from human patterns — your training data distribution assumptions from pre-2025 are obsolete.