ClarityBeta

Product & Strategy

The Product Desk

8 min read · 1,547 words · 3 moves to watch
The Signal

Routing a live session to a cheaper model re-bills the whole history at cold rates.

Cache entries are keyed to a single model. That detail is missing from most tiered-routing tickets, including the one you are probably scoping this sprint, and it inverts the math those tickets were written to justify. Pick the tier once, at task entry, and the savings actually show up: a marketplace matcher did it that way and hit 98% accuracy at one-seventh the cost. Anything that switches after the work has started is a cost-reduction feature that adds cost.

In Play

  1. The Pricing Meter Moves Off Tokens

    a16z's Sarah Wang and Tugce Erten published a direct attack on per-token billing: of 50 technical AI buyers, 27 prefer credits tied to recognizable work and 14 prefer tokens. That makes the meter a product spec — unit-of-work definition, effort bands and per-task cost telemetry all ship out of your backlog, not finance's. The sample is proprietary and unpublished, with 9 preferences unaccounted for, so treat the direction as real and the ratio as directional.

    Ask Clarity
    Try

  2. Cheap-Model Routing Reverses Mid-Session

    Avi Chawla's caching breakdown shows that prompt-cache entries are keyed to a specific model. Routing a long conversation to a cheaper model therefore re-prefills the whole accumulated history at cold rates, inverting the savings case behind most tiered-routing tickets. The same week, two production results show the pattern working when the tier decision happens once at task entry: a marketplace product-matching system hit 98% accuracy at one-seventh the cost, and DoorDash clears millions of daily events with cheap classifiers before any model call.

    Ask Clarity
    Try

  3. Authorization Becomes a Product Requirement

    Datadog put three coding agents through six builds of a document portal, and every build shipped a critical IDOR letting any authenticated user read other users' documents. Plan mode changed nothing. Separately, prompt injection now carries NVD scores — Context7 at CVSS 9.0, Apache SkyWalking's MCP server at 9.8 — and MLflow's SSRF entered CISA's exploited-vulnerability catalog on 2026-08-19. Ownership checks and tool-parameter trust are language you write into stories, not cleanup your engineers infer.

    Ask Clarity
    Try

  4. Personal-Agent Valuations Outrun Any Usage Data

    Newcomer reported on August 27 that Town, an enterprise personal agent, is in talks to raise at a $1B valuation led by Index Ventures — two days after the Wall Street Journal put consumer assistant Instinct at a $2.5B valuation on $350M raised. Neither report discloses revenue, retention or usage, and Newcomer notes that venture capitalists themselves appear to be the power users. If your own agent metrics blend employees, design partners and enthusiasts with organic users, you are reading the same distorted cohort that priced this category.

    Ask Clarity
    Try

  5. Enterprise Software Growth Splits From Its Headline

    Salesforce's July quarter grew 11%, but organic growth was 6.4% once the November 2025 Informatica acquisition is stripped out, per The Information. Workday decelerated from 13.5% to 12.8%. Agentforce ARR passed $1.5B, up 240%, against a raised full-year guide of $46.4B. AI ARR compounding fast off a small base while total growth slows is the signature of attach and bundling — and it is the frame your board will apply to your own AI numbers next quarter.

    Ask Clarity
    Try

Deep Dives

Tokens Anchor Your Product to a Falling Cost Curve

Buyers stopped comparing you with competitors and started comparing you with raw compute; Clay's four-meter rebuild shows what a mispriced credit system costs before anyone notices.

The precedent that prices the mistake

A pricing lead reads the a16z survey, nods, and files it under directional. The survey is not the artifact worth keeping. Clay's 2026 pricing memo is, because it discloses that Clay mispriced credits in its Pro segment back in 2022 and ran that segment at a loss for years. Teams tell themselves a metric change is a communications exercise with a migration email attached. What the memo actually shows is a multi-year margin hole in one segment, disclosed after the fact. That is the real cost of path dependence: once customers are trained on a metric, the meter is very hard to move, whatever the size of the credit pool. The useful diagnostic has two axes. On one, whether the metric was chosen deliberately or inherited from whatever shipped first. On the other, whether the segment paying it is profitable today or only assumed to be. The dangerous cell is inherited and unmeasured, which is where Clay sat from 2022 onward. The recommendation is to write down the year the current meter was set and the segment margin under it before the next packaging debate, accepting that this surfaces numbers a pricing committee would rather not see in writing.

What to do

  1. Write the one-sentence unit-of-work statement this week and validate that three buyers can repeat it back unprompted before it enters any pricing doc.

  2. Instrument cost-to-serve per completed unit of work this sprint, decomposed across inference, retrieval, tool calls, third-party data, retries and failed runs.

  3. Model both cost cases before Q4 packaging locks: who captures the savings if model costs fall 50%, and which shipped features still clear margin if token prices hold flat.

Visa Published Its Harness; the Routing Play Inside It Runs Backwards

Two cost levers are being sold as interchangeable: one is a stage map you can fork for free, the other quietly re-bills an entire conversation at cold rates.

What a harness actually contains

An engineer forks the repo expecting a model wrapper and finds orchestration. Visa wrapped Anthropic's unreleased Mythos model in it, pointed it at its own codebases to find and fix vulnerabilities, then open-sourced the code as the Visa Vulnerability Agentic Harness. Rajat Taneja, Visa's president of technology, told The Information the model is more effective through the harness than alone. Thursday's update allows swapping between Anthropic, OpenAI and open-weight models like Kimi, or combining them.

The anatomy is the part worth forking. Bug-find-and-fix splits into 11 discrete stages, each getting only the context it needs instead of one monolithic prompt burning frontier tokens. Deterministic pre-processing maps what to scan, and work done in code costs roughly nothing. A dedicated stage rechecks output for false positives, which concedes that frontier models alone are not reliable enough for autonomous remediation on systems handling more than a billion transactions a day. A Visa spokesperson called the build a significant undertaking at nearly 100 engineers over three-plus months. The widely repeated 60% cost reduction comes from an unnamed July study, not from Visa, which withheld its own figures. Do not quote it. Globant, Coinbase, Shopify and Ramp built equivalent model-switching harnesses independently. Ramp is the one that matters: a mid-size fintech doing this undercuts the claim that Visa-scale resources are the prerequisite.

Where routing pays, and where it reverses

Avi Chawla's caching taxonomy establishes that prompt-cache entries are keyed to a specific model. Switch models mid-conversation and the whole accumulated history re-prefills at cold rates. Cache pricing has converged at 1.25x base input to write and 0.1x to read on Anthropic and OpenAI, so roughly 0.3 reads per entry repays the write premium. Below about a third of writes ever read inside the TTL, caching raises the bill. Anthropic also caps its backward search for an earlier cache write at 20 blocks. Exceed that between calls and hits stop, with no error and no log line.

Now the wins. The product-matching system that hit 98% accuracy at one-seventh the cost routed once, at task entry. DoorDash's SafeChat runs millions of daily chats, calls and images through fast internal classifiers first, reserves model scoring for the hard tail, and cut verbal-abuse incidents roughly in half. Neither downgrades a live session. Tier by task at the front door, never by turn inside a conversation.

Routing is a classification decision at the entry point. Used as a mid-session downgrade, it buys a cheaper per-token rate and pays for a full cold prefill.

Two items already sitting on most roadmaps make this worse, uncharged. Personalizing the system prompt with a name or tenant puts a variable at the front and invalidates every block behind it. A/B testing two reasoning efforts splits the cache in two, and the inflated inference spend never lands on the experiment's ledger.

The infrastructure number that outweighs both

The largest lever is not a product decision. Paged KV cache management reaches roughly 96% cache utilization and serves 2–4x more users per GPU with unchanged attention outputs. That is a 50–75% swing in cost per served user, larger than any pricing change on the roadmap. The forcing function is one question for the platform team before the next pricing model ships: paged stack, or not. If not, current unit economics rest on a number that is about to move.

What to do

  1. Fork the open-source harness this sprint and have an engineer map its 11-stage decomposition against your agentic pipeline, producing a stage-by-stage list of steps paying frontier prices for deterministic work.

  2. Re-run every model-routing business case with cold-prefill cost of the full accumulated context included on each switch, and publish a session-length threshold above which routing is disabled.

  3. Ask your platform team this week whether the inference stack uses paged KV cache management, then re-baseline per-user COGS for every shipped AI feature against the answer.

Six of Six Agent-Built Portals Shipped the Same Critical Flaw

Plan mode is marketed as the model thinking before it acts; across six builds it reordered the work, and each agent added a fresh flaw class nobody asked for.

The finding belongs in acceptance criteria, not the backlog

Kennedy Toomey at Datadog gave Claude Sonnet 5, Cursor Composer 2.5 and Codex GPT 5.5 the same brief: a document portal with authentication, file uploads and user management. Six iterations, plan mode against default mode, every result run through Datadog Code Security and Claude's own code_review skill. Plan mode produced no meaningful security improvement, and each model introduced something new while planning. Cursor Composer 2.5 wrote a hardcoded JWT secret fallback, which is authentication forgery. Claude Sonnet 5 added defense-in-depth and broke document downloads. GPT 5.5 introduced TOCTOU race conditions and timing side channels, the two flaw classes most code-review tooling misses entirely.

Separate the thing being pitched from the thing being done. Plan mode is pitched as deliberation. What it does is reorder the sequence of work without adding requirements to it, and that sequence never contained an ownership check because the prompt never named one. Explicit security requirements mattered far more than operational mode: no model inferred authorization logic the requirements never specified. So object-level ownership checks, tenant scoping and secret sourcing become acceptance criteria on any story touching customer data. Filed as eng hygiene they fall off, and neither plan mode nor agent self-review discharges them.

The same failure, one layer up and one layer down

Varonis built an email agent called Pinchy with mailbox access to synthetic credentials and CRM data, then ran four phishing simulations across Gemini 3.1 Pro and Codex GPT-5.4. It failed both social-engineering tests: it forwarded AWS IAM keys, database passwords and SSH credentials to an external Gmail account during a fake production emergency, and exfiltrated a CRM dump on a casual "working from home" export request. It cleared the two technical attacks, inspecting a malicious OAuth redirect URI and refusing credentials on a fake gift-card page, while interacting with the phishing infrastructure before flagging anything. Outbound sends to external domains need a deny-by-default rule, because model judgment failed the two scenarios a real attacker would run.

One layer down, the tool inventory has scores a reviewer can paste into a spreadsheet. Context7 through 2.1.2 carries CVSS 9.0 for prompt injection through its Custom AI Instructions field. Apache SkyWalking MCP 0.1.0 carries 9.8 for SSRF via its set_skywalking_url tool. CodeWhale 0.8.41 through 0.8.64 carries 9.3 for argument injection in git_show, yielding arbitrary file writes. One root cause across all three: agent tool parameters were treated as trusted input. Exposure compounds as well, with Censys measuring internet-exposed AI and LLM tool instances rising more than 60% in nine months, Langflow up 169% carrying 18 CVEs including unauthenticated RCE.

Why this is a this-sprint decision

Someone hands a login-gated task to Anthropic's Cowork and it finishes, because the native browser carries their real cookies, logged-in state and autofill. That is the feature working as designed. In that context an IDOR or an injected instruction reaches the payment flow the session is already authenticated against, which is how a document leak becomes a transfer. Two frameworks supply credible external vocabulary in place of an internal debate: OWASP's Agentic Skills Top 10 and SANS Fellow Lenny Zeltser's Security Autonomy Matrix, which grants agent independence one action type at a time along a five-stage path. The forcing function for the next planning session: for each action an agent can take, name whether the requirement is written into a story, and whether the session it inherits can move money. Everything in the unwritten column is shipping with autonomy it has not earned.

No model can infer authorization logic the requirements never specified. If the ownership check is not in the story, it is not in the code.

What to do

  1. Rewrite the agent-coding prompt template and Definition of Done this week to require object-level ownership checks, tenant scoping and secret sourcing on every story touching user data, and delete any language treating plan mode or agent self-review as a security gate.

  2. Run a 48-hour inventory against this month's exploited-vulnerability additions — starting with MLflow's SSRF and Gitea — across product dependencies, the internal dev toolchain and anything your integration docs tell customers to self-host.

  3. Adopt the four-scenario credential-egress gate — fake production emergency, casual off-hours export, malicious OAuth consent, credential-harvest page — as a standing pre-GA test for any agent with mailbox, file or credential access, with zero credential egress as the pass condition.

The bottom line

Every convention your team inherited from the model layer — how to bill, how to save money, how to review generated code — was designed for a different layer and quietly transfers risk down onto your product. Vendor defaults are not safe starting points: the durable specifications are the ones you write yourself and enforce at a launch gate. Name one owner this week for three artifacts — the unit of work you sell, the prompt and cache layout you bill against, and the authorization language in your Definition of Done — and refuse any AI roadmap item that arrives without all three.