The Engineer

The benchmark compares two different things

An Expo founder made the strongest objection, calling the move "more of a political decision than a technical one" and going after the benchmark: Shopify compared an old-architecture React Native app against a fresh native rewrite. Expo's commercial model depends on React Native's enterprise credibility, so the objection is self-interested. It is also right on methodology. Any 12-week greenfield build wins on startup time, stability and binary size against a six-year-old app, whatever runtime it targets. The measurements in the companion post are real. They measure the rewrite.

The stated argument is an economics claim, not a performance one: coding agents made building the same feature twice cheap enough that cross-platform's central economic justification stopped paying for itself. That is a statement about Shopify's own throughput. Making the same trade requires a measured agent-throughput number from the team making it. The closest published proxy in the same reporting is Next.js closing 1,500 GitHub issues in a month with agents involved. No reopen rate published.

The dependency graph generalizes

Shopify's post names Skia, FlashList and Restyle. All three sit on hot paths. FlashList sits in the render path of every long list that uses it, and Skia underpins most custom RN rendering. Maintenance funding for all three just moved off the runtime they target. The failure shows up transitively, inside an upgrade window. React Native 0.88 is at release candidate, so the sequencing is fixed: pin exact versions, mirror the repositories, and write down the named upstream maintainer for each package before taking the RC.

Shopify is acquiring Tailwind Labs the same week, with the open source staying MIT-licensed under the same team. Read that as a reallocation: funding moves toward web styling infrastructure Shopify owns.

Where the migration hours actually go

Two migration datapoints in the same reporting agree. Discord's New Architecture migration had only 14% of tickets on the migration itself; the remaining 86% was long-tail work. Evil Martians moved a site from Gatsby to Astro in under 9¾ days of execution, after substantial preparation.

The framework conversion is the only schedulable part of a rewrite. Estimate the long tail separately, and expect it to dominate.

The toolchain moved the same week

React 19.3 graduated Fragment Refs and <ViewTransition> to stable. Every wrapper element that exists only to hold a ref becomes deletable code, and the layout-animation dependency goes up for re-evaluation. Independent transition rendering is the free win: transitions no longer block each other, so the "filter input freezes while the results pane hydrates" class of bug loses its mechanism with no code change.

oxc's Rust port of the React Compiler is the item that needs a decision. It took a 1000+ file React Router app's compile step from 14.3s to 0.81s, and it already compiles patterns React Compiler 1.0 skips. Broader pattern coverage means different memoization boundaries. Engineers running oxc locally while CI runs the Babel plugin will produce environment-dependent render counts. Those reproduce only on the machine that produced them. Switch the whole toolchain or none of it; per-developer preference is the one configuration that guarantees the bug. React DevTools 8.0 also removed the Timeline profiler in favor of the browser Performance panel, so any perf triage runbook that says "open the React Timeline" is already invalid.

Where the 245 seconds actually went

Attribution matters more than the total. On a cold scale-to-zero endpoint, one job spent 171.89 seconds booting to serve 0.56 seconds of inference. Median cold start across three clean trials was 245 seconds for a 1.5B model on a 4090. Nothing was cached. Every wake pulled the vLLM worker image, then downloaded the base model and all three adapters from Hugging Face before the engine warmed. So the time went to image pull and weight download, not GPU initialization. That puts a third-party registry inside the critical request path, on a code path nobody tests and nobody can page. The fix is boring and already available. Bake base weights and hot adapters into the worker image, or pre-seed them on a network volume, then enable snapshot-style boot acceleration. Runpod's own published case cut a 32B model on 2×H200 from 324s to 91s with four config changes, steady-state throughput unchanged.

The denominator inverted the answer

Under a 5-minute rotating load at 1 request/second, the three-separate-endpoint layout won on billed worker-seconds and lost badly on requests served. It spent most of the window cold-starting and completed far fewer requests. Most of the ones it did complete waited over two minutes.

Any cost metric without a completed-request denominator will eventually recommend the architecture that answers the fewest requests.

Capacity pooling causes the loss, not memory pressure. Separate endpoints are separate scaling pools. An idle worker holding the extraction adapter cannot pick up a SQL-generation request, so that request boots a fresh worker while a paid GPU sits idle. A shared endpoint grows the pool with traffic, not with the number of fine-tunes, and MAX_LORAS lets multiple adapters share a single batch, so mixed-variant traffic batches instead of serializing.

The storage math is the boring half

A rank-8 LoRA is about 40 MB. Merge it into a 7B base and it becomes another 15.2 GB checkpoint. Three variants: 45.6 GB merged, against one base plus roughly 120 MB of adapters. One hundred variants: 1.5 TB against ~19.3 GB. At rank 16 across all seven projection modules, the measured adapter files came out near 70 MB each. At a hundred per-customer fine-tunes, that multiplier shows up in gross margin.

The two hard walls, and the honesty caveats

Every adapter must derive from the same base model, so shard by base and run one shared endpoint per base. Adapters also compete with KV cache for the same budget. 15.2 GB of base weights plus ~4 GB of adapters leaves roughly 60 GB of an 80 GB card for cache. That constraint is why request-time adapter resolution exists for long-tail catalogues at all.

Treat the comparative result as a hypothesis to reproduce, not a benchmark to cite. The warm-path latency numbers were never published. The medians covered five requests on a 1.5B model. The platform relationship is disclosed, and framing 245s as "an unoptimized baseline, not a property of the platform" is exactly what a sponsor would want said. The raw measurements survive that bias. The interpretation should be re-run on the stack that will pay for it. Two hygiene items keep those numbers honest. Discard any trial where a worker reports throttled, because boot time then conflates with capacity waiting. And assert on GET /openai/v1/models before trusting anything: if only the base model appears, the adapters silently did not load and every response came from the wrong artifact.

Why the sequencing is the finding

A reservoir simulator has a property most business systems lack: correctness is numerically expressible. You can diff old and new output within a tolerance and let that harness be the reviewer. So the parity rig came first, then agents were used for comprehension — over a hundred of them documenting a Fortran 77 codebase nobody alive fully understood — and only then for translation, behind human gates. Most teams do this in the opposite order and discover their oracle problem after they have thousands of lines of plausible output.

Be honest about the scope before you cite it internally: 40,000 of 300,000 lines is 13% of the system, described as a first sprint covering core functionality. That is a strong proof of concept, not a turnkey modernization story, and any vendor citing it as evidence of autonomous migration is overreading it. The transferable rule is narrower and harder: if you cannot construct a mechanical oracle for a system, that system is not an agent target yet. Say that out loud in planning.

Where the sources agree, and where one pushes back

Two independent threads land on the same structural point. Coding agents both write and remediate code, which collapses the independence assumption underneath pull-request review — reviewer and author draw from the same distribution. At the same time the security "left" boundary moved off the repository and onto developer endpoints, where agents run with broad privileges outside anyone's control. Two firehoses, one triage queue.

The useful dissent comes from Kästner, surfaced by Pointer: teams are weakening review because AI writes the code, while implicitly mourning a pre-AI review process that was never a gold standard — most of it was line-level nitpicking with weak defect-detection evidence. Take both claims together and the conclusion is not "restore human review." It is that the artifact being inspected is wrong.

Context assembly is a build artifact with a silent failure mode

DongYun Kang's finding is the highest-value technical claim in this material: AI review quality is a function of context size. An under-contexted reviewer does not fail loudly — it returns a confident, well-formatted approval. You get the ceremony of review with none of the coverage, and the problem worsens automatically as your repository grows, with no alert firing.

Which means context assembly deserves the treatment any other CI stage gets: deterministic, versioned, budgeted, observable. Emit a manifest of what actually entered the window — diff, call-graph slice, related tests, schema, relevant postmortems. Alert when a diff blows the budget and split the pull request instead of reviewing it degraded. Then validate the whole thing against ground truth you already own: run mutation testing, feed the surviving mutants back through the reviewer as synthetic pull requests, and record recall. That converts "we have AI review" into a number you can defend.

An AI reviewer starved of context does not throw an error. It sends you an approval.

One number in circulation deserves discounting: the claim that 81% of bugs leave no trace in the code diff is sponsored copy with no methodology, sample size, or codebase profile. Discount the figure, keep the shape — agent-generated code is usually syntactically clean and wrong in composition: broken cross-module invariants, mishandled retry idempotency, quietly changed error semantics. None of that is visible in a diff, which is why linters, static analysis, human eyeballs and AI diff review all share one blind spot. The control-theory framing makes the arithmetic plain: when your actuator has orders of magnitude more bandwidth than your sensor, the system does not converge, it accumulates unverified state. There are exactly two levers — raise sensor bandwidth, or shrink the step size.