The Product Desk

Why a fluent plan misses the page

The agent opens the live page and reaches for a button that is not on it. Then it reports back that the step is done. The mechanism here is not model quality. The failing planner was finetuned without ever seeing the target sites, so it emitted steps that were linguistically correct and referentially empty, naming buttons, fields and flows the page did not contain. The executor has no veto. It receives a step, tries to satisfy it, reports back. Fluency is what makes this expensive. A plan that reads like competent human instruction passes every review a team actually runs, including the demo.

Granularity is the other half, and it is a spec problem the product team owns rather than a training problem. The paper's planner writes "input New York as the arrival city" instead of "input the arrival city", because the vaguer version leaves the executor guessing which value belongs in the box. One step covers exactly one unit of work. "Search for the product in the search box" is a step. A single click is too small. "Analyze the search results" is not a step at all. It hands the reasoning back to the executor the planner exists to relieve. Planner prompts are reviewable copy, not config. They belong in the same review queue as user-facing strings.

Read the progression as a build decision

The middle two rows carry the product lesson. The same architectural change is a downgrade or an upgrade depending entirely on whether the planner was trained against the environment it will act in. That is a data question, not a modeling question. It is answerable before a single percent of traffic moves.

Where the old loop decays

The ReAct baseline's weakness is structural, not a tuning gap. Every thought, action and observation is appended to one growing prompt and nothing ever leaves, so a failed search from step three is still competing with the original objective at step twelve. Teams tell themselves the agent gets confused on hard tasks. What session replays show is degradation as a function of task length, and no amount of prompt polish fixes an append-only architecture. Plan-and-Act's executor strips HTML it no longer needs after each action, and replanning replaces the failed step instead of accumulating it.

Instrument tokens-in-context per step against task success by step count, and context pollution stops being an argument in a design review and becomes a number on a dashboard.

The dial, and the thing that is not a moat

Dynamic replanning produced the largest single increment, +10.31 points, at roughly one extra planner call per executor step. The paper's authors flag that cost as unresolved and suggest letting the executor decide when to replan. Treat that as a tiering decision rather than a default. Always-on replanning for enterprise and high-value tasks, executor-triggered replanning on free and high-volume paths. The unsolved half is where a differentiator lives, because every competitor is busy copying the solved half.

The caveat worth holding. All of this is public, cheap and reproducible by backend engineers with no ML hire. It moves the quality number and it does not defend it. What cannot be copied is the environment-grounded training data and the frozen eval set drawn from your own task distribution, which is exactly why the ungrounded planner failed. The forcing function before the next sprint plan is to name which of those two exists today, and staff the one that does not.

The concession that makes this worse, not better

An analyst opens the retention chart, sees the softest surface bending down for two straight quarters, and writes "macro" in the notes column. That may be true. Resale demand is cyclical and secondhand apparel has taken macro hits before. The concession is not a defense, it is the problem: the company had no decomposition to put on the table, so an outside read filled the vacuum and became the record. When "macro" is the residual bucket in your own retention analysis, the explanation of your own numbers has been pre-delegated to whoever publishes first.

The counter-move is arithmetic and boring. Decompose the last two quarters of retention on the softest surface into traffic and mix effects on one side, and activation, experience and brand-pull effects on the other. Name an owner for each driver. If at least 80% of the delta does not map to a named, owned driver, the exercise did not happen. The goal is not to be right about macro. It is to be able to answer inside a day.

Where definitional attacks actually originate

The Bending Spoons thesis pairs "flattered by a non-standard definition" with "material control weaknesses" against a $28.3B market capitalization, for a roll-up whose assets include AOL and Vimeo. That first phrase is a product artifact, not a finance one. Definitions of active user, engaged session, attach rate and organic-versus-acquired growth start in a product analytics spec. Finance inherits them. The Street repeats them.

Here is what teams tell themselves happened: nothing changed. Here is what actually happened: someone changed a session timeout, re-scoped what counts as an activated account, or folded an acquired product's users into the organic cohort during a dashboard migration, and nobody wrote it down. Two quarters later the trend line is structurally different and no one can reconstruct why. With a dedicated Financial Reporting and Accounting Unit sitting inside the SEC's Division of Enforcement, undocumented metric drift has graduated from data hygiene to disclosure exposure.

The pattern the two halves of this briefing share

The agent evidence and the short evidence are the same failure at different altitudes. A fluent artifact passes internally because nobody holds it against a comparison that could contradict it. A plan that reads correctly. A growth number that trends correctly. In the agent case the missing comparison is a frozen eval set. In the demand case it is an attribution with named owners. Both take roughly two weeks to build. Both are worthless if the build starts after the question arrives.

"Macro headwinds" is no longer a defensible explanation for a demand miss, because someone outside your company will publish the product diagnosis before you do.

The version you can defend

One document. Every externally reported product metric, its precise definition, and a change log with dates and rationale covering the trailing eight quarters. Then run the moat stress test across the roadmap: tag each item harvest, defend or rebuild against the question of what a frontier model does to it within 18 months. The bears are pricing legacy consumer assets as melting ice cubes, and thin wrapper features will get the same treatment. Reallocating one sprint out of "harvest" this quarter is the cheapest way to find out whether the defend list is real, and the tradeoff is that the sprint comes out of shipping something a customer already asked for.

What the screen actually measures

An analyst sorted the comp set by growth rate this week and found Cloudflare sitting below Palantir. That is the entire moment. The short case does not claim the company is shrinking. It claims the growth was purchased at poor marginal economics: 12% incremental operating margins at more than 35x revenue, while growing slower than Palantir. Separate the thing being pitched from the thing being done. Pitched: durable compounding worth a premium multiple. Done: each additional dollar of revenue drops twelve cents to operating income. The forcing function is whether the next dollar of revenue costs less to serve than the last one. If it does, the multiple has something underneath it. If it does not, 35x is a bet on a leverage curve that has not yet appeared in the incremental margin.