The Engineer

Why containment shipped before reliability did

Z.ai moved GLM-5.3's Terminal-Bench 3.0 score from 4.6 to 28.3, and DeepSWE from 46.2 to 66.9, on GLM-5.2's unchanged base model. Extended post-training and reinforcement learning did all of it. Two consequences follow. Agentic competence is a post-training artifact now, so behavior can move by a factor of six between point releases of the same base. Pin versions and re-run evals on every release. Then there is the other reading of the same figure: 28.3 is the top open score, which means terminal agents still fail roughly 72% of tasks. Policy gateways, credential vaults, egress controls, and human mid-task takeover are blast-radius primitives. They are what gets built for a component that succeeds under a third of the time and whose actions have real side effects.

The cold-start number decides your capacity model

Alibaba's sub-800ms claim is not attributed to a runtime or an image. The omission is load-bearing, because these boundaries are not interchangeable.

Cold start is dominated by image size and storage class, not the runtime alone. A Chrome plus Playwright image is not a busybox image. That single measurement decides whether a sandbox gets allocated per request or drawn from a warm pool. Warm pools reintroduce exactly the cross-task state leak the isolation existed to prevent. A genuinely fast cold path is worth real engineering effort.

The demand is arriving from the tier that fails review

The revenue proving computer-use agents is coming from the local OS tier, because that is the only tier that survives the messy long tail of real desktop work. It is also the tier with ambient credentials: the user's SSO session, VPN, cloud credential files, local git identity. The actions look exactly like the human's own in any audit log. Three topologies exist, and choosing among them is the architecture decision: API tool-calling with scoped revocable tokens; a sandboxed remote browser or VM with brokered per-task injection; local OS control. Default-deny the third pending review. Insist any evaluation measure per-task success rates on your workflows, not OSWorld or WebArena.

What to build regardless of which platform wins

1. A distinct principal per agent, vault-injected secrets rather than environment variables, and a per-agent egress allowlist with cloud metadata endpoints blocked.
2. Per-task credentials with a TTL measured in minutes. Model artifacts pulled by digest with signature verification, never by mutable tag.
3. An append-only audit record written before each side effect, plus screenshot replay. Logging afterward yields a history of successes and no record of the action that hung.

One caution the platform pitches skip: full trajectory logs contain system prompts, tool arguments, and intermediate reasoning. That means secrets and PII. That log needs access control and a retention policy on day one, not after the first audit. Keep the adapter at the protocol boundary. If agents speak a stable protocol to the execution layer, swapping frameworks is a per-agent refactor instead of a platform migration.

The containment layer built for an agent that fails 72% of tasks still works for one that fails 20% — the prompt tuning done for a specific checkpoint does not survive the next post-training run.

Read the data-efficiency number, not the lift

GenRec's +1.6% relative MRR is offline lift. At most companies that sits inside the noise band of the experiment platform. Online validation was four weeks on about 10% of traffic. Do not build a business case on reproducing it. Build it on the 40x reduction in training data, which is what changes cold-start, new-market, and new-vertical launch economics, the scenarios where years of interaction logs do not exist. The training shape makes that reduction structurally believable: an infrequently updated domain-adapted foundation model, then frequent ranking-specific post-training with a reward-weighted ranking loss. The base already encodes catalog and behavioral semantics, so the ranking head learns far less from scratch. The expensive adaptation stops blocking fast iteration.

The serving decision underneath removes the usual objection. Pack the full candidate set into one context, take a single forward pass, and the model becomes a scoring function rather than a generator. There is no autoregressive decode loop to destroy p99. Anyone who killed an LLM-ranker proposal on decode latency has lost that argument. Once serving is prefill-dominated, token count is the cost function. That is why the compression result transfers to routes already running today with no model change.

Budget the constraint layer in the same build

Netflix documented the failure modes, and they are structural rather than tuning bugs: over-recommending globally popular content, hallucinating out-of-catalog titles, and ignoring nuanced business constraints. A conventional ranker encodes catalog membership and business rules structurally. A generative ranker does not. Catalog-membership validation, exposure floors for long-tail inventory, and hard post-scoring business-rule filters belong in the build, not a follow-up ticket.

Speculative decoding is a tuning surface, not a flag

The vLLM team benchmarked five drafting methods on AMD MI300X, 288GB of VRAM per card: Native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark. Reported range was 1.27x to 2.87x throughput across workloads including HumanEval. The ceiling is not the finding.

Operationally, acceptance-rate-by-position telemetry belongs on the inference dashboard and a throughput regression test belongs in CI. Without both, a model upgrade silently inverts the gain and nobody notices until the bill arrives. The benchmarking burden recurs with every upgrade. Staff it rather than assuming it away.

Why this is budget defense, not craft

The Information reports server makers already quoting customers a roughly 17% increase on Nvidia AI chips, driven by memory prices Nvidia is passing through rather than absorbing. Neoclouds pass it on fastest, having no captive silicon and thin margins. On-prem buyers eat the inflated bill of materials once, at purchase. The arithmetic is unusually clean. Recover about 20% throughput and the increase is cancelled outright and permanently. A signed reservation only defers it. The sources agree on the mechanism and diverge on relief. Alternative accelerators look cheaper on paper, but Nvidia gained inference share through the shortage, so treat portability as a priced option that stays cheap, not a migration to plan this year.

Ignore the MRR delta. Prefill-only single-pass scoring plus a two-thirds cut in context tokens is what makes LLM ranking cheap enough to ship.

Look at the call path, not the headline

A third-party platform exposed a request handler that mutates your public surface area, meaning search visibility. The authentication on that handler was a plausible assertion of rights ownership. Most abuse and legal-removal flows verify by email domain, by web form, or at best by a document upload reviewed by a human under time pressure. There is no cryptographic binding between the request and the entity it names, and no notification to the party being impersonated. Muddy Waters' public comment was that "a lot of roads seem to lead to 1xBet," a Sportradar customer. This is single-sourced reporting: treat the attribution as unproven and the mechanism as confirmed.

Enumerate every equivalent handler pointed at your organization

• Google legal removals and Search Console
• The DMCA agent inbox
• Your domain registrar and DNS provider
• CDN and hosting abuse desks
• App store report and takedown flows
• Package registry ownership transfer at npm, PyPI, crates.io
• Social platform brand-impersonation reports
• Certificate issuance

For an engineering organization the registry row is the one that reaches production. A package your services resolve at install time can change owners through a support process you do not operate, verified by nobody internal, monitored by nothing you built. Every row above is a write path into your public identity with an external operator on the other end.

The detection gap costs a cron job

Ship a public-surface canary. Daily reachability and index checks on the top-N critical URLs. Content hashes compared against the last known good. An alert when something changes with no corresponding deploy. That deploy-correlation filter is the part that keeps the signal quiet enough to page on, and index APIs are eventually consistent, so allow for lag before escalating. Then make forgery harder at the transport layer: DMARC at p=reject with aligned SPF and DKIM across every sending domain, plus look-alike domain monitoring off Certificate Transparency logs and registrar feeds. Human reviewers at abuse desks pattern-match on domains. Make the confusable ones visible the moment a certificate is issued.

Every abuse-report and takedown form pointed at your company is an unauthenticated write endpoint against your public surface — instrument it like one, because the attacker already knows it exists.

The same pipeline is your best vendor early-warning system

The generalizable pattern is that the observable artifact leads the authoritative record, and the artifact is scrapeable. Roster-page diffing caught a COO departure at a $3.95 billion company with no filing and no press release, weeks or months before formal disclosure. The pipeline is boring: scheduled fetch, DOM-stable selectors, content hashing, human triage queue. Point that same change detection at critical vendors' status pages, API changelogs, docs, pricing, terms, and SDK release notes. Vendors edit those pages because they must, long before they email anyone.

The variant worth a sprint reconciles what a supplier says against what it files. Core Scientific management stated repeatedly on its earnings call, including when asked directly, that AMD was providing "full credit support." An analyst flagged that as not reconciling with the 10-Q language. For anyone consuming hosted GPU capacity or colocation, the counterparty credit structure behind that supplier is an availability dependency, and it is almost certainly unmodeled. Pull both artifacts. Where they diverge is where capacity disappears in a bad quarter.

The architecture is more defensible than the number

Graft's benchmark is not budgetable. The design is still worth reading. It writes a tree-sitter-derived graph of the codebase into the repository as linked markdown: deterministic operations, no API key, no network, no vector database to keep warm. That deletes an entire operations surface. No embedding drift, no reindex jobs, no stateful service paging someone at 3am. The trade is weaker semantic recall. Reported gains are 42% fewer tokens, 46% fewer tool calls, 60% less task wall-clock.

Two failure modes ship with it. A generated graph committed to the repository goes stale after every refactor, conflicts across parallel branches, and adds diff noise to reviews. The worse one is that it becomes a prompt-injection surface. The agent treats those files as authoritative ground truth about the codebase, and any contributor with pull-request rights can edit them. The shippable version regenerates in CI on main, fails the build on divergence, marks the files generated in .gitattributes, and sits behind CODEOWNERS. Same treatment for any repo-committed agent memory.

What "blocked on a better model" actually means

Eric Topol is a cardiologist who calls Demis Hassabis "a hero of mine" and secured a blurb from him. He still called the 60 Minutes claim that AI will cure all diseases within a decade "hype," on the grounds that there is no precedent for curing diabetes, Alzheimer's, or heart disease in five to ten years. Drop the media frame and the argument is about pipeline topology. AI compresses candidate generation, the one parallelizable stage. Trials and review stay wall-clock bound. Stanford's Daphne Koller named the omitted bottlenecks: data quality, causal inference, and validation.

Those three map one-to-one onto enterprise failure modes. Upstream pipelines are dirtier than the model card assumes, and the model launders that dirt into confident output. The model found a correlation, the product turned it into an action, and nobody wrote down which is which. With no closed loop confirming the output was correct, quality regressions arrive as customer escalations rather than a failed CI gate. A better base model solves none of the three.

Logs are the only audit surface on offer

Closed labs obfuscate reasoning tokens. The trace that comes back is therefore a produced artifact, not an execution log. It cannot prove why a decision was made. It does not diff reliably across versions. No auditor will accept it as evidence. Request and response capture, tool-invocation logs, and output-level eval results are the whole audit story under local control.

Security claims get the same read. GLM-5.3 reportedly leads CyberGym vulnerability discovery at 84.5%, narrowly ahead of Claude Mythos 5 at 83.8% and GPT-5.6 Sol at 83.6%. Its advantage reverses sharply on ExploitBench and ExploitGym. All self-reported, all unreplicated. Discovery outrunning exploitation predicts high-volume, low-validation findings against dependencies. Z.ai's models reportedly filed 2,436 findings across 269 open-source projects, 1,097 rated critical or high. Triage queues need to absorb volume, not sophistication.

Two patterns worth stealing this quarter

• A five-university study finds agent skills work as procedural anchors that stabilize execution, not as knowledge injection. They fail from poor retrieval precision in large candidate pools, or from guidance applied to incompatible contexts. The standard instinct is to add more skills. Past that point it is actively harmful. Cap the pool and measure retrieval precision.
• Agent Lightning v1.0 lets the deploy-time harness manage the environment loop during post-training, collapsing the training simulator and the production harness into one codebase in roughly 3,500 lines. Reported +14.6% on SWE-bench for a coding agent. The real work is retokenization and dynamic sample counts. The payoff is no train/serve environment skew: the harness you measure with is the harness you ship.

When the vendor's reasoning trace is unverifiable and its benchmark is its own, your eval suite and your request logs are the only ground truth that will ever exist.