Science & Analytics

The Scientist

The Signal

Two inference-layer disruptions landed the same day

If you're making inference architecture decisions this quarter, your comparison matrix just lost a row and gained a column. Port one agent eval to Gemini native tools this week; pause any Mojo/MAX commitments until Qualcomm clarifies cross-silicon support.

In Play

  1. Inference Stack Double Disruption

    Gemini 3.5 Flash makes computer-use a built-in tool (no scaffolding), while Qualcomm acquires Modular for $3.9B, coupling Mojo/MAX to a chip vendor. Agent frameworks and silicon-agnostic compilers — two layers you may depend on — are both under pressure simultaneously.

    Ask Clarity
  2. Industrial-Scale Model Extraction Is Now Documented

    Anthropic accuses Alibaba of running 25,000 fake accounts generating 28.8M queries against Claude — ~1,150 queries/account, comfortably below naive rate-limit thresholds. Literature confirms 1–10M well-chosen queries can distill competitive models. Per-key RPS limits are useless against this attack shape.

    Ask Clarity
  3. AI ROI Tracks Process Redesign, Not Model Quality

    a16z-cited study of 515 high-growth startups: firms that redesigned workflows around AI generated 44% more use cases, ~2x revenue at the top 5%, and consumed ~40% less capital. Corroboration: Accenture's FCF multiple collapsed from 30x to 6x — the market pricing in that consulting-led 'bolt-on AI' doesn't deliver.

    Ask Clarity
  4. GDPR Enforcement Expanding to ML Pipelines

    Two independent sources converge: GDPR at 10 is explicitly targeting AI training data, embeddings, and inference logs. Regulators extending Article 5/6 scrutiny beyond user databases into model artifacts. Teams without documented lineage from raw event to embedding to prediction face regulatory debt, not just technical debt.

    Ask Clarity
  5. Physical AI & Bits-to-Atoms Capital Rotation

    Robotics/Physical AI hit ~$16B across ~500 deals in Q1 2026 — 2x deal count and 4.5x deal value vs 2021–2025 baseline. Now #2 by category in top-100 private companies, ahead of fintech. Data platforms built for clickstream and SaaS telemetry face pressure-test from multimodal sensor, video, and trajectory data within 24 months.

    Ask Clarity

Deep Dives

Your Inference Architecture Just Got Two Shocks — Here's the Decision Matrix

Two simultaneous shifts to the same layer

Google shipped native computer-use in Gemini 3.5 Flash this morning. The release lands on the inference and agent-execution layer that many teams rebuilt late in 2025. A separate event in the same layer, the Qualcomm acquisition of Modular, arrived the same day and pulls in a different direction.

Gemini 3.5 Flash ships native computer-use

Google released computer-use as a built-in tool across browser, mobile, and desktop surfaces. No scaffolding. The model itself emits structured browser and desktop actions, which collapses the agent-framework layer (Browser-Use, Claude Computer Use wrappers, LangGraph orchestration) into a model primitive. The tradeoff is framework complexity for tighter latency and lower cost per step, with reduced observability.

The thing this release doesn't tell you is how it performs on real workloads. No benchmarks were published. Easy demos will look fine, and multi-step web tasks probably land where everyone else lands, at first. The validation path is mechanical: port 20–50 representative tasks from the current agent eval onto native tool-calls and compare task success rate, mean steps to completion, p95 latency, and cost per successful task.

Qualcomm acquires Modular for $3.9B

Modular's pitch was vendor neutrality: write once in Mojo, run efficiently across NVIDIA, AMD, Intel, and emerging accelerators. The prior cases rhyme. Intel/Habana, NVIDIA/OctoML, AMD/Nod.ai. Cross-silicon parity erodes quietly once a chip vendor controls the roadmap. Qualcomm calling it 'silicon-agnostic' in the press release is the standard line; the empirical question is whether AMD and Intel backends keep getting optimization cycles in the next four quarterly releases.

A chip vendor doesn't buy a compiler company to keep it neutral.

The combined implication

A stack that looked like Mojo/MAX for inference optimization → custom agent framework for browser automation → tool-call accuracy eval harness is now under pressure on all three layers at once. Even with a single-layer dependency, the decision window is this quarter.


What to do this sprint

RuntimeSilicon coverageRoadmap risk2026 recommendation
vLLMNVIDIA-first, AMD growingLow (OSS)Safe default
TensorRT-LLMNVIDIA onlyLowSafe if NVIDIA-committed
SGLangNVIDIA, AMDMediumBest for structured generation
MAX (Modular)Multi (claimed)HighPause new bets

For the agent layer: run the Gemini native-tool eval in parallel with the existing harness for one release cycle. Define success by end state, not path taken. The old tool-call-accuracy metric breaks once the model drives the screen itself. If rankings reshuffle, and expect at least one position change, the native-tool number is the one that matches user experience.

What to do

  1. Spike a 20–50 task benchmark comparing Gemini 3.5 Flash native computer-use vs. your current agent framework — measure task success, steps, p95 latency, cost per task

  2. Freeze any new Mojo/MAX commitments and redirect those engineering weeks to a vLLM-vs-TensorRT-LLM-vs-SGLang bake-off on your served traffic

  3. Add live-site trajectory evals (end-state scoring, not path-matching) to your agent harness in parallel with existing scripted tool-call tests

The 28.8M-Query Extraction Playbook: Build Your Detector Before Someone Runs It on You

The attack is now documented at industrial scale

Anthropic publicly accusing Alibaba of running 25,000 fake accounts to issue 28.8 million queries against Claude is the first public confirmation of model-extraction-at-scale against a frontier API. The numbers matter more than the accusation. About 1,150 queries per account sits comfortably below most naive abuse thresholds. Per-key rate limits will not catch that. Most existing telemetry will not flag it either.

The academic literature (Tramèr et al., Krishna et al.) puts 1–10M well-chosen queries in the range where a competitive distillation target is recoverable for non-trivial tasks. At 28.8M you are well past that, especially on narrow task slices where a high-quality student model is the realistic outcome. The thing the headline number doesn't tell you is how concentrated the queries were on the slices that actually transfer. That is the variable that decides whether the attack worked.

Per-key rate limits don't stop a 25,000-account distillation attack. Distribution-level monitoring does.

The defensive architecture

The blind spot is structural: per-account telemetry is useless when the campaign is distributed across 25K accounts. What measures the right thing:

  • Prompt embedding entropy per account cohort — real users produce diverse, contextual queries; extraction campaigns produce systematically varied but statistically distinguishable patterns
  • Account-age-to-query-complexity ratio — extraction accounts ramp to sophisticated queries faster than organic users
  • Cross-account clustering — supposedly independent accounts querying the same regions of capability space at coordinated rates
  • Account-creation-to-first-query latency — synthetic accounts follow different temporal distributions than genuine signups

Complementary defenses

Detection alone is not sufficient. Layer these:

  1. Output watermarking or canary tokens on high-value generative endpoints, which gives you a way to identify leaked outputs downstream
  2. Response perturbation for accounts with anomalous embedding distributions, degrading outputs enough to reduce distillation fidelity without affecting normal users
  3. Query-budget throttling at the cohort level, not per-key, but per-cluster of accounts with similar behavioral signatures

If a model is served via API, including internal APIs consumed by partner teams, this attack shape applies. The Anthropic disclosure is effectively a free red-team report on a documented production attack pattern. The open question is whether the defense gets instrumented before someone runs the playbook against another set of endpoints or after.

What to do

  1. Build an extraction-detection prototype: instrument per-account prompt-embedding entropy, account-age-vs-complexity ratio, and cross-account behavioral clustering on your inference API telemetry

  2. Add a synthetic red-team test to model-serving CI: simulate 1K accounts × 100K queries with adversarial prompt diversity and require detection within a query-budget threshold

  3. Deploy output watermarking or canary tokens on your highest-value generative endpoints

The Mapping Problem: 515 Startups Prove Process Redesign Beats Fine-Tuning

The study and its implications

A treatment/control study of 515 high-growth startups, cited via a16z, isolates what may be the cleanest empirical signal on AI adoption this quarter. The intervention was information — showing firms how peers reorganized production around AI. Not access to better models, not more compute, not more talent. Just discovery of what's possible when you redesign rather than retrofit.

Results at the tails: treatment firms generated 44% more AI use cases, roughly 2x the revenue at the top 5%, and consumed ~40% less capital. The gains compound at the distribution extremes, not the mean. The 2x gap narrows to 50% at the top decile — this is heavy-tailed, which means the winners win disproportionately.

The causal question

The honest read: the correlation is real, but the causal share is almost certainly smaller than 40%. Startups willing to redesign workflows also tend to have founders who pick leaner go-to-market motions, hire differently, and move faster on adjacent decisions. The study can't separate these. If the migration captures a quarter of the reported gain on your numbers, it probably still pays off.

Market corroboration

Accenture's FCF multiple collapsing from 30x to ~6x — roughly one-third of its long-term average — is the market's verdict on consulting-led AI rollouts. The 'bolt-on AI transformation engagement' model is being priced as failing to deliver. One firm isn't the whole sector, but a five-fold compression is hard to explain with macro alone.

AI ROI in 2026 tracks process redesign, not model quality. The firms that treat it that way generate 2x the revenue with 40% less capital.

What this means for your team

The paper reframes AI adoption as discovery over a latent space of business processes. Replicating an existing workflow 'but with AI' is a local minimum. Most RAG-on-existing-docs initiatives sit in this retrofit bucket. The treatment group started from the business outcome and ended up with a different process entirely.

Practical implications:

  • Instrument discovery, not just deployment. Track 'use cases discovered per quarter' and '$ impact per shipped redesign' — not just models deployed or eval scores. The data locates the bottleneck upstream of the model.
  • Benchmark team sizing against AI-native ratios. YC W20–F24 cohort data shows AI startups start smaller, stay smaller, run flatter. Stripe data shows >2x more $1M+ solopreneurs in 2025 vs 2023. Revenue-per-headcount benchmarks inherited from 2019-era SaaS are obsolete.
  • Audit retrofit vs. redesign in your current project portfolio. Any project framed as 'use an LLM to do step X of an existing workflow' matches the control group. The treatment group redesigned the entire flow.

What to do

  1. Run a 2-week use-case mapping sprint: catalog every business process where AI could intervene upstream (not as drop-in replacement), score by $ impact × feasibility, commit to redesigning top 3 end-to-end

  2. Add 'use cases discovered' and 'headcount-per-$1M-revenue' to your team's quarterly metrics, benchmarked against AI-native YC cohort distributions

  3. Audit any active consulting-led AI initiative against the redesign criterion — are they designing new processes or wrapping existing ones? Escalate findings to leadership

The bottom line

Your inference stack took two hits today — Gemini made computer-use a model primitive (your agent framework is now a wrapper around a wrapper) and Qualcomm bought Modular for $3.9B (your 'silicon-agnostic' compiler bet just got vendor-locked) — while Anthropic publicly documented a 28.8M-query model-extraction campaign that flew under standard rate limits at 1,150 queries per account. Rebuild the eval harness, pause Mojo/MAX, and instrument distribution-level extraction detection before someone runs the Alibaba playbook on your endpoints.