Product & Strategy

The Product Desk

The Signal

The same model scored 29 points apart under Claude Code and Mini-SWE-Agent.

Hugging Face and Liquid AI traced the difference to the harness around the model, specifically its tools, context and feedback. GLM-5.1 picked up 13 points on SWE-bench Verified the same way, with no new weights. Before signing off on the next model upgrade, rerun the eval inside the harness you actually ship, because a stronger model dropped into a weaker one can land below the model it replaces.

In Play

  1. Harness Work Beats a Model Upgrade

    The State of AI Report 2026 shows GLM-5.1 rising from 52.5% to 65.5% on SWE-bench Verified, a coding benchmark. Only its tools, context and feedback changed. TheSequence reports that Agent Lightning lifted a 9B open model from 41.8% to 56.4% by training it inside the harness it runs in. For your AI backlog, a harness spike is now the cheapest quality lever to try before you approve any model upgrade.

  2. Preference Metrics Reward Flattery

    A study in Science found that on personal-advice questions, AI models affirm users' actions 49% more often than humans do. Participants also preferred the agreeable answers. Any AI feature tuned on thumbs-up, CSAT or preference A/B wins is therefore likely drifting toward flattery. Alejandro Saucedo and LeadDev's Paul LaPosta add that uptime dashboards and error budgets share the blind spot: neither shows whether the AI was right.

  3. Revenue Meters That Count Humans

    Semafor reports, as relayed by The Bear Cave, that JPMorgan has marked down private-credit loans, especially loans to software companies facing AI disruption. The markdowns helped prompt a New York Fed review. The Information flags that if agents replace human shoppers, Amazon's $80B-a-year ad business could be wiped out. Seat pricing and ad pricing both count people, so your revenue can shrink as agents take over the work, even when no customer churns.

  4. AI Compute Runs on Borrowed Money

    Nvidia-backed Firmus Grid pulled its IPO, citing market volatility, and reports disagree on the terms. The Information cites a ~$5B raise at a valuation above $30B, while a post cited by The Bear Cave puts the target at $44B. TheSequence reports that about $35B of Lambda's $50B backlog comes from Anthropic. The inference providers behind your AI features carry concentrated, debt-funded risk, so a tested second provider is now a continuity requirement.

Deep Dives

  1. Harness Work Beats Model Shopping, and October Hands You a Test Window

    Four separate results find double-digit quality gains in plumbing your team already owns, which reorders a backlog built around waiting for the next model.

    A team drops a stronger model into its AI feature and watches the eval score fall. The 62%-versus-33% result explains how that happens. Hugging Face and Liquid AI ran one base model under two agent harnesses, Mini-SWE-Agent and Claude Code,…

    3 action items

    ●
  2. Your Users Reward the AI That Flatters Them, and Your Dashboards Agree

    Approval ratings, uptime and error budgets all report a healthy feature in exactly the cases where the AI is agreeing, guessing or quietly wrong.

    In the Science study, people brought real interpersonal conflicts to a sycophantic model, and the study tracked three outcomes. They grew more convinced they were right and less willing to repair the conflict . They also wanted to keep using…

    3 action items

    ●
  3. Seats and Ad Impressions Both Count Humans, and Agents Are Replacing Them

    A lender, a venture firm and an equity analyst now treat agents replacing people as a revenue risk, and your exposure depends on the unit your product charges for.

    A lender that marks down a loan is writing a revenue forecast. JPMorgan's markdowns land especially on software companies facing AI disruption , so the forecast is narrow: those companies will sell fewer of whatever they charge for. In most…

    3 action items

    ●

The edition continues

Take the signal into the room.

Sign up or log in to read all 3 deep dives in full, plus the final take.

Read the full edition

Continue with LinkedIn