Science & Analytics
The Scientist
AT&T bought a 56% coding cost cut with a 2% quality drop it never measured.
Forty percent of queries on open models is not forty percent of tokens, and it is further still from forty percent of dollars. The cheap tier summarizes code the frontier models are still writing. An unverified mean can also hide a steep regression concentrated in the hardest slice of traffic, which is where rework hours actually accrue. If the routing mix in your own cost model looks like this one, the savings and the quality figure are being measured on different populations.
In Play
Production Routing Finally Has a Reference Number
AT&T's VP of AI, Mark Austin, told The Information that a LiteLLM complexity router cut AI coding costs 56% for a stated 2% quality decline. Open-weight models now serve 40% of employee queries at 45 billion tokens per day. Deep dive below on why the cost cut transfers and the quality figure doesn't.
Ask ClarityThe Aggregate Passed, the Missing Arm Failed
All 33 blood-glucose prediction models in one review looked equivalent in aggregate, yet every one ran roughly 6 mg/dL worse for Type 1 than Type 2 patients. Separately, an SSRN working paper reports that generative AI raised pupils' homework scores 18% while their closed-book exam scores fell 20%. Deep dive below on the shared defect and the release gate that catches it.
Ask ClarityMLflow Joins the Actively Exploited Column
CISA confirmed active in-the-wild SSRF exploitation of MLflow servers. DataDog separately analysed N4D Mesh Controller, a Linux strain scanning for internet-exposed MCP servers. The same security reporting timed GitLab CVE-2026-19478 to exploitation 48 hours after proof-of-concept code dropped. A monthly patch cadence on internet-adjacent ML control-plane hosts is not defensible against that window. The exploitation chain and the inventory it implies are in the deep dive.
Ask ClaritySemi-Structured Columns Go Native in Two Engines at Once
DuckDB 2.0 and Iceberg v3 both shipped native VARIANT types on Parquet binary encoding, queryable from Spark SQL via variant_get, with timestamps, decimals, and binary values preserved inside the column. That retires the JSON-string-plus-cast-on-read pattern in model-input logging, where a silent cast failure surfaces three weeks into a training run. Netflix separately published a greater-than-99.99% dual-run agreement bar before retiring a legacy pipeline, which transfers verbatim as a feature-pipeline migration gate.
Ask ClarityCompute Scarcity Is Priced in Time, Not Kilowatt-Hours
ERCOT's June 2026 interruptible-load rule compresses interconnection from 5-7 years to 12-18 months for battery-buffered data centers that accept curtailment, and FERC has written to six other grids urging adoption. On a 1GW build, five years of energy is 4-9% of total cost of ownership while accelerators are roughly 70% of capex. That inverts the standard efficiency roadmap: model FLOPs utilisation and preemption tolerance are the cost levers, and joules per token is a carbon metric.
Ask Clarity
Deep Dives
- ●
The 56% Replicates. The 2% Doesn't.
AT&T's cascade is the first enterprise cost/quality frontier anyone can copy, but the savings arrive as a mean while the risk concentrates in the hardest decile of your traffic.
Start with the denominator Forty percent of queries is not 40% of tokens, and it is a long way from 40% of dollars. Summarization prompts are short. Agentic code generation is long-context and output-heavy. The disclosure carries its own tell:…
3 action items
- ●
Five Reported Wins, One Missing Arm
Subgroup gaps, tool-removed capability drops, self-grading judges, and argmax with free ground truth available are the same measurement defect wearing four different costumes.
The mechanism generalises past glucose Type 1 physiology is more volatile, which means conditional variance differs by cohort . A loss function minimising average error will happily trade accuracy on the smaller, harder, higher-variance subgroup for accuracy on the easy…
3 action items
- ●
Your RAG Index Is Laundering the ACLs
A high-privilege crawler plus one shared vector index turns row-level access control into cosine similarity, and the failure is measurable long before it becomes an incident report.
The failure class DLP does not cover In March 2026 an internal AI agent at Meta produced a Sev 1 by surfacing sensitive company and user data to employees with no entitlement to it. Not a jailbreak, not prompt injection.…
3 action items
The edition continues
Take the signal into the room.
Sign up or log in to read all 3 deep dives in full, plus the final take.
Read the full editionContinue with LinkedIn