The Scientist

Two innovations landed this week that challenge your fundamental assumptions about long-context processing

NVIDIA's Nemotron 3 Super introduces LatentMoE — a genuinely novel mixture-of-experts approach that compresses each token's representation to 1/4 its size before routing, enabling 22 active experts per token at the compute budget of 5-6. Combined with a mamba-2/transformer hybrid architecture, the result is a 120B total parameter model with only 12B active per token that delivers:

• 442 tok/s throughput (59% faster than gpt-oss-120b at 278, 66% faster than Gemini 3.1 Flash-Lite at 266)
• 91.75% RULER accuracy at 1M tokens where gpt-oss-120b collapses to 22.30%
• $0.30/$0.80 per million tokens (input/output) with 1M in/1M out context window

The model was pretrained natively in NVFP4 (4-bit floating-point) on Blackwell GPUs — not post-hoc quantized. This means optimal inference requires Blackwell hardware, and cross-hardware portability is an open question. The 3-stage RL pipeline (verifiable-output tasks → GitHub issue solving with test execution → RLHF) is a replicable fine-tuning recipe worth studying.

RLMs: The Context Window Is Dead, Long Live the REPL

MIT researchers propose Recursive Language Models that treat input text as a persistent variable in an external Python REPL. Instead of stuffing documents into context, the model writes code to query, slice, and filter text programmatically, then recursively decomposes sub-problems.

The most striking result: RLM-GPT-5 maintains ~50% accuracy at 1M tokens on tasks where stock GPT-5 scores ~0%. A 32K-context Qwen3-8B goes from 0% to 14% when wrapped in an RLM loop. The claim is documents totaling 11M+ tokens are tractable.

If a 32K-context model can handle 11M tokens through programmatic context management, we've been over-investing in context window scaling and under-investing in agent-mediated retrieval.

Critical gaps in the RLM work: no latency overhead analysis for the recursive agent loop, no token cost multiplier vs. single-pass inference, and no failure mode characterization. These are essential for production feasibility assessment.

The Combined Implication

These two approaches attack long-context from opposite directions. Nemotron scales the native window with architectural efficiency; RLMs bypass the window entirely via code generation. If you're building RAG or document QA systems, you now have two fundamentally different design patterns to evaluate — and both outperform the standard "embed → cosine similarity → top-k → stuff" approach.

Three independent results prove agent performance is pipeline-bottlenecked

NVIDIA's ProRL Agent makes the most uncomfortable claim this week: fully decoupling rollout from optimization nearly doubled Qwen 8B's SWE-Bench Verified score from 9.6% to 18.0%, with similar gains for 4B and 14B variants. The mechanism is straightforward — decoupled rollout achieves higher GPU utilization by eliminating the serialization bottleneck. This is purely an infrastructure architecture change, not a new reward function or training objective.

The implication is stark: many published agent benchmark results may be confounded by infrastructure quality rather than reflecting genuine algorithmic improvements. If your RL training pipeline couples rollout and optimization on the same compute, you may be benchmarking your infra limitations, not your model's ceiling.

Cursor's Continuous RL: Training Meets Serving

Cursor announced "real-time RL" that uses live production inference as training data: user accept/reject/edit behavior feeds reward signals, enabling model checkpoint redeployment every 5 hours. This is an online RLHF loop where the "H" is implicit.

Missing details undermine full evaluation: What's the reward model architecture? How is reward signal noise handled (users accept bad code out of laziness)? What distribution shift management is applied? The compounding data moat argument is theoretically sound, but the 5-hour figure could be best-case rather than steady-state.

Selective Feedback: 10x Compute Reduction

A trending paper claims selective feedback training matches full RL accuracy at 10x less compute by updating only on mistake steps — essentially hard example mining applied to temporal credit assignment. A second paper shows automated system prompt rewriting yields up to 30% reasoning improvement without retraining. A third enables 70B fine-tuning on a single consumer GPU via CPU-GPU memory splitting.

The "up to 30%" framing is a red flag — it means best-case on the easiest benchmark. Still, automated prompt optimization at zero retraining cost pays for itself in days if even 10% materializes.

Before you try a fancier reward model or more training data, try a better pipeline. The Qwen 8B jump from 9.6% to 18.0% on SWE-Bench is entirely from infrastructure, not algorithmic changes.

Chroma Context-1: Retrieval Gets Its Own Model Tier

Chroma released Context-1, a 20B parameter agentic search model trained on 8,000+ synthetically generated tasks. It separates retrieval from generation entirely — decomposing queries into sub-queries, searching iteratively across multiple turns, and selectively discarding low-relevance results as context fills. Claims: frontier-level retrieval at 10x inference speed at a fraction of the cost. Benchmarks are thin — no dataset specs or statistical significance tests — but the architecture pattern (separate the search model from the generation model) is directionally compelling.

The LiteLLM story is already covered — these are the NEW threats that landed this week

While LiteLLM dominated headlines (and our previous briefings), three genuinely novel attack vectors quietly emerged that target different parts of your AI stack. None are caught by traditional AppSec tooling.

1. Context Hub MCP Poisoning: Fake Docs → Fake Dependencies

Andrew Ng's Context Hub, which feeds API documentation to coding agents via MCP servers, has zero content sanitization. Researcher Mickey Shmueli found that 58 of 97 closed PRs (59.8%) were merged without review. His proof-of-concept: planted fake PyPI package names in Plaid and Stripe documentation. A coding agent consuming this context would suggest installing malicious packages.

This is dependency confusion one layer up — instead of poisoning the package registry, you poison the documentation that tells the AI which packages to recommend. Lower barrier, harder to detect, and it scales through every MCP-connected coding agent consuming those docs.

2. LLM-to-SQL Injection: Your Output Is the Attack Vector

Researchers demonstrated that LLM-integrated databases create a fundamentally new attack surface where SQL injection flows through model output, not user input. The chain against SQLite-backed systems:

1. Guardrail bypass via crafted prompt injection
2. Schema discovery — the LLM generates queries revealing table structures
3. Data exfiltration — unauthorized queries execute because they appear to be legitimate model output

Traditional input sanitization is defending the wrong perimeter. The fix requires a mindset shift: treat LLM output as untrusted input. Add validation between model output and query execution — allowlisted SQL operations, parameterized query patterns, and anomaly detection on generated queries.

3. OpenClaw: Agentic AI Accumulates CVEs at 200x the Rate

OpenClaw, an autonomous AI agent with default shell execution and file system access, accumulated 104 CVEs in just 18 days — claimed 200x faster than LangChain or Ollama across their entire lifetimes. Root cause of CVE-2026-27001: the working directory path was embedded as a plain string in the LLM system prompt, and attackers injected instructions via Unicode bidirectional markers in directory names.

The pattern is clear: AI tools are simultaneously expanding your attack surface and sitting at credential boundaries where traditional security tools are blind.