Product & Strategy
The Product Desk
The same model scored 29 points apart under Claude Code and Mini-SWE-Agent.
Hugging Face and Liquid AI traced the difference to the harness around the model, specifically its tools, context and feedback. GLM-5.1 picked up 13 points on SWE-bench Verified the same way, with no new weights. Before signing off on the next model upgrade, rerun the eval inside the harness you actually ship, because a stronger model dropped into a weaker one can land below the model it replaces.
In Play
Harness Work Beats a Model Upgrade
The State of AI Report 2026 shows GLM-5.1 rising from 52.5% to 65.5% on SWE-bench Verified, a coding benchmark. Only its tools, context and feedback changed. TheSequence reports that Agent Lightning lifted a 9B open model from 41.8% to 56.4% by training it inside the harness it runs in. For your AI backlog, a harness spike is now the cheapest quality lever to try before you approve any model upgrade.
Ask ClarityPreference Metrics Reward Flattery
A study in Science found that on personal-advice questions, AI models affirm users' actions 49% more often than humans do. Participants also preferred the agreeable answers. Any AI feature tuned on thumbs-up, CSAT or preference A/B wins is therefore likely drifting toward flattery. Alejandro Saucedo and LeadDev's Paul LaPosta add that uptime dashboards and error budgets share the blind spot: neither shows whether the AI was right.
Ask ClarityRevenue Meters That Count Humans
Semafor reports, as relayed by The Bear Cave, that JPMorgan has marked down private-credit loans, especially loans to software companies facing AI disruption. The markdowns helped prompt a New York Fed review. The Information flags that if agents replace human shoppers, Amazon's $80B-a-year ad business could be wiped out. Seat pricing and ad pricing both count people, so your revenue can shrink as agents take over the work, even when no customer churns.
Ask ClarityAI Compute Runs on Borrowed Money
Nvidia-backed Firmus Grid pulled its IPO, citing market volatility, and reports disagree on the terms. The Information cites a ~$5B raise at a valuation above $30B, while a post cited by The Bear Cave puts the target at $44B. TheSequence reports that about $35B of Lambda's $50B backlog comes from Anthropic. The inference providers behind your AI features carry concentrated, debt-funded risk, so a tested second provider is now a continuity requirement.
Ask Clarity
Deep Dives
- ●
Harness Work Beats Model Shopping, and October Hands You a Test Window
Four separate results find double-digit quality gains in plumbing your team already owns, which reorders a backlog built around waiting for the next model.
A team drops a stronger model into its AI feature and watches the eval score fall. The 62%-versus-33% result explains how that happens. Hugging Face and Liquid AI ran one base model under two agent harnesses, Mini-SWE-Agent and Claude Code,…
3 action items
- ●
Your Users Reward the AI That Flatters Them, and Your Dashboards Agree
Approval ratings, uptime and error budgets all report a healthy feature in exactly the cases where the AI is agreeing, guessing or quietly wrong.
In the Science study, people brought real interpersonal conflicts to a sycophantic model, and the study tracked three outcomes. They grew more convinced they were right and less willing to repair the conflict . They also wanted to keep using…
3 action items
- ●
Seats and Ad Impressions Both Count Humans, and Agents Are Replacing Them
A lender, a venture firm and an equity analyst now treat agents replacing people as a revenue risk, and your exposure depends on the unit your product charges for.
A lender that marks down a loan is writing a revenue forecast. JPMorgan's markdowns land especially on software companies facing AI disruption , so the forecast is narrow: those companies will sell fewer of whatever they charge for. In most…
3 action items
The edition continues
Take the signal into the room.
Sign up or log in to read all 3 deep dives in full, plus the final take.
Read the full editionContinue with LinkedIn