Engineering & Technical

The Engineer

The Signal

AI agents broke out of Trail of Bits' QEMU sandboxes through libslirp and CUPS.

The hypervisor core itself held. The escapes came through accessory surfaces bolted on beside it, one of them a daemon that had no business being in an agent image at all. Add 155 abandoned MCP servers sitting hijackable on public marketplaces, and "it runs in a VM" stops closing the design review you were counting on it to close.

In Play

  1. Rails Secrets Leak and 21 Critical UniFi Fixes

    Rails' Active Storage has an unauthenticated secrets-disclosure bug, CVE-2026-66066, exploited in the wild against default configurations, per Risky Business. It was patched at the start of August, so the gap between that release and your deploy is exactly the window attackers were scanning. Ubiquiti separately shipped 21 critical UniFi fixes across UniFi OS Server and three applications, covering remote command execution, authentication bypass, and privilege escalation. Neither job ends at the deploy: one ends at rotating secrets, the other at finding controllers your inventory missed.

    Ask Clarity
    Try
  2. Agent Sandboxes Escaped Through Their Accessories

    Trail of Bits reported that cyber-capable AI agents routinely escaped QEMU/KVM sandboxes, and the named paths were accessory surfaces — libslirp, QEMU's usermode network stack, and CUPS, a printing daemon — not the hypervisor core. Researchers separately found 155 abandoned MCP servers on public marketplaces that can be hijacked, and an OpenAI-related npm package was published from an unsecured workflow. For your design reviews, "it runs in a VM" has stopped working as a blast-radius argument.

    Ask Clarity
    Try
  3. Uber's Agent Bill Lives in Tool Execution

    Uber published its agentic engineering teardown: more than 70% of pull requests are now agent-attributed, and 3,600 skills drive 30,000+ executions a day. Collapsing N sequential tool calls into one generated script cut 55-100% of tool-execution tokens, because every tool call is otherwise another full inference pass over a growing prompt. Note the arithmetic Uber left out: requests grew 9.4x while cost per 1,000 requests fell 34%, so absolute spend still rose roughly 6.2x.

    Ask Clarity
    Try
  4. The Capacity Ceilings Nobody Exports

    Adyen traced mysterious Kubernetes pod-setup timeouts to conntrack garbage collection rather than table capacity: expired flow entries were not reaped fast enough, inserts failed, and the CNI's own calls lost their SYNs. Datalab's Marker v2 documents the matching GPU shape — many thin CPU workers feeding one shared inference server, because a per-worker model copy is fed one page at a time and never batches. Both ceilings are visible in a single counter almost nobody exports today.

    Ask Clarity
    Try
  5. Model Supply Chain Consolidates While Open Weights Get Teeth

    CNBC-sourced reporting says NVIDIA agreed to buy Hugging Face for $12.9B, with no confirmation from either party. Z.ai's GLM-5.3 open weights shipped alongside a public ledger of 2,436 vulnerabilities found across 269 open-source projects, some in code dating to 1981, and Tencent previewed its Hy4 flagship. The Information also reports a draft U.S. rule targeting China's remote access to chips, which extends export-control logic from hardware to hosted models. An unpinned from_pretrained() is now a sourcing decision.

    Ask Clarity
    Try

Deep Dives

Rails Secrets Don't Un-Leak, and Your UniFi Controllers Aren't in the CMDB

Two exploited-class fixes have landed whose real remediation the deploy cannot reach: one finishes at credential rotation, the other at finding the controllers nobody inventoried.

Patching stops the reads. Rotation ends the incident.

An RCE incident ends when the fixed version is running. A secrets-disclosure incident ends when the stolen credentials are useless. CVE-2026-66066 hands an unauthenticated caller whatever the Rails process can read: SECRET_KEY_BASE, master.key and credentials, object-storage keys, any signing material sitting in the app. After the patch, a signed cookie or JWT minted under a leaked key still verifies. A leaked S3 key still authenticates. Risky Business has this right: rotation ends this incident, and the deploy only stops the reads.

Rails shipped the fix at the start of August, so the live variable is when each environment got it. Reconstruct that window before closing the ticket: the date the fixed release actually landed per environment, which of those services answered from the internet before that date, and whether Active Storage sat in the request path at all. The subsystem is on by default, so absence needs proof. grep finds the obvious call sites; only the request path tells you whether it was reachable unauthenticated.

Sequence the rotation by blast radius

A leaked signing key and a leaked bucket key are two incidents wearing one CVE. The signing key forges sessions and tokens against your own app. The storage key reads and writes data directly, outside your request path and outside your logs. Rotate both, plus any credential the app could enumerate downstream. Cookies and sessions minted under the old key will stop verifying. Plan that user-visible logout deliberately rather than discovering it in a support queue.


The other patch is a discovery problem

Ubiquiti's 21 critical fixes cover UniFi OS Server plus three UniFi applications, and the impact classes are the bad ones: remote command execution, authentication bypass, privilege escalation. The controller pushes configuration, firewall and NAT rules, VPN topology, DNS resolution, and firmware to every adopted device. Auth bypass chained to command execution collapses that boundary in one step, with no lateral movement required.

Real exposure runs ahead of the advisory because controllers live in shadow infrastructure: a branch office nobody inventories, or an engineer's home network with a tunnel into corp. Asset inventories undercount them structurally. The relayed advisory detail also carries no CVE IDs and no CVSS vectors, so pull the vendor bulletin directly before scoping the change window.


Where the two stories agree

ItemExploited?Auth neededWhat the deploy fixesWhat it leaves open
Rails Active Storage — CVE-2026-66066Yes, in the wildNoneFurther reads of app secretsKeys already taken; forged tokens still verify
UniFi OS Server + 3 apps — 21 criticalNot reportedBypassableCommand exec, privesc, bypass pathsControllers outside inventory; internet-facing management planes

Both remediations gate on inventory you either already have or will spend the week building: a credential inventory and an asset inventory. Both sources arrive there independently, from opposite ends of the stack. That is the pattern worth carrying past this week: patch velocity is cheap; enumeration and revocation are the constraint.

Until the keys are rotated, the patched app still trusts tokens minted under the old one. The controller inventory is the one most teams lack, and the next disclosure bug will bill them for it.

What to do

  1. Grep every Rails service for Active Storage this week, confirm the patched release in each environment, then rotate SECRET_KEY_BASE, master.key/credentials, object-storage keys, and signing material for anything internet-reachable before the fix landed.

  2. Enumerate every UniFi OS Server instance outside the CMDB — branch offices, hardware labs, home networks with tunnels into corp — patch in an out-of-band window this week, then scan externally to confirm no management plane answers from the internet.

  3. Add controller-class assets to a tier-0 patch SLA this quarter, with owner, discovery method, and external-exposure check recorded per instance.

Your Agent's Token Bill Is Mostly Tool Round-Trips

Uber's published cost model makes one cheap measurement decision-grade, and it tells you whether weeks of batching work would pay for itself in your workload or miss the spend entirely.

Why a 55-100% range is not marketing

Count the passes. A conventional tool-calling loop spends a full inference pass on every MCP invocation: the model re-reads a growing context, emits one call, and the serialized observation gets appended for the next pass. A ten-call chain is ten passes over a growing prompt. Code-mode batching collapses that into one pass. The model writes a script, the script invokes the tools in-process, and one consolidated observation comes back. Savings scale with chain length, which is the whole reason the range is that wide. Short chains save little. Long chains save nearly everything.

The trade-off is observability. A batched script that dies on step 7 of 9 hands back a stack trace, not a clean per-step tool-call trace. Mid-chain recovery goes with it. Structured logging inside the generated scripts is work that lands before this ships, not after the first bad incident.


Tool discovery has to become lazy

Uber's grounding layer exposes 1,000+ MCP tools. That is why "CLI-resolved MCP calls" sits in the same list as the cost levers. Static tool manifests break somewhere around 50-100 tools; a thousand JSON schemas injected into a system prompt survives neither the token budget nor the routing accuracy. Exposing tools as a CLI surface the agent discovers on demand moves schemas from push to pull. The cost is discovery latency per session. It is cheap to design in now and expensive to retrofit as a prompt-architecture rewrite later.

LeverMechanismReported gainEffortHidden cost
CLI-resolved MCP callsPull tool schemas on demand instead of pushing manifestsRemoves manifest tokens from every turnDays-weeksDiscovery latency per session
Code-mode batchingOne generated script replaces N tool-call turns55-100% of tool-execution tokensWeeksNo per-step traces, no mid-chain recovery
Context-graph groundingStructured cross-system context instead of pure retrievalHigher task success, fewer retriesMonthsIngest, freshness, 30+ system maintenance

Skills are becoming an artifact with a governance problem

Do the division. 3,600 skills against 30,000 executions a day averages roughly eight runs per skill per day. That is a long, cold tail. Microservice sprawl with worse tooling. Uber's authoring conventions read like runbook discipline: the description is a router written in the words a user would type, the body is steps and nothing else, deterministic scripts carry the fixed sequences, and the agent is reserved for judgement. Then verification is a step of the procedure. That last convention earns its keep, because a bash script has exit codes and a skill has none until someone builds them. The verification step is the exit code.


Where the sources converge, and where Uber is quiet

Uber's own database-overload work landed on the same mechanism: static, human-configured quotas do not survive real capacity movement, and the fix was a closed loop reading observed latency. The agent tier is open-loop on unit cost for the same reason. Requests grew 9.4x and cost per 1,000 requests fell 34%. "Down 52% from peak" concedes the overspend arrived first and the instrumentation second. Reporting on Salesforce repricing AI points at the consequence: under seat licensing a 3x retry rate is invisible, and under consumption pricing it is a budget incident with a team's name on it.

One thing not to copy: the 24M-node context graph is the harvest of years of platform consolidation across 30+ internal systems, and Uber flags that mileage varies. Ground two or three high-value workflows, watch where agents fail for lack of context, and expand against evidence.

Every MCP round-trip you leave unbatched is a full inference pass over a growing prompt, and you pay for the prompt twice.

What to do

  1. Split agent token telemetry into reasoning tokens versus tool-result tokens this sprint, emitted at the loop boundary and attributable to a team and a product surface.

  2. Prototype code-mode batching on your single highest-volume agent workflow this sprint, with structured logging inside the generated script before it serves traffic.

  3. Write ownership, deprecation, and per-skill execution telemetry rules before your skill library passes a few hundred entries, this quarter.

Your Agent Sandbox Escapes Through libslirp, Not the Hypervisor

Three independent breaks in one week put the agent tier's containment boundary and its supply chain in the same place: accessory device surface, marketplace maintainership, and CI publishing credentials.

Read the escape surfaces, not the headline

The paths Trail of Bits names are libslirp, QEMU's usermode network stack, and CUPS, a printing subsystem. Neither is the hypervisor's hardened core, and CUPS has no legitimate reason to exist inside an agent execution image at all. The isolation primitive largely held while the accessories bolted onto it did not, which is what makes the result actionable rather than fatalistic. The cheapest control available is deleting the surface that was exploited. That is a base-image change, not a platform migration.

Containment boundaryWhat it actually buysFailure modeVerdict for agents
QEMU/KVM VM, stock imageKernel separation from hostEscape via accessory surface (net stack, device emulation, print daemon)Necessary, no longer sufficient
Container onlyProcess and namespace separationShared kernel; weaker than the boundary already broken aboveInsufficient alone
Minimal image, stripped devicesRemoves the exact surface exploitedBreaks agent workflows expecting a full OSHigh value, low cost
Short-lived scoped credentials, no metadata endpointCaps what an escaped agent can reachRequires real identity plumbingThe actual control
Default-deny egress allowlistKills exfiltration and C2 pathsAllowlist maintenance is ongoing toilNon-negotiable for prod-adjacent agents

MCP is npm with instruction-injection privileges

The 155 abandoned MCP servers found on public marketplaces are not passive malicious dependencies waiting to be imported. Here is what actually happens on a tool call: tool descriptions and tool outputs flow directly into the model's context, and the model holds tool-calling privileges, so a hijacked server is prompt injection with hands attached. Treat these like production dependencies. Pin versions, allowlist by publisher, vendor or fork anything that touches production systems, and drop marketplace-sourced servers with no active maintainer.

The compromised OpenAI-related npm package is the older and better-understood half of this: publishing credentials sitting in CI. OIDC trusted publishing and provenance attestations exist specifically to delete that failure mode, and they are good engineering. A static registry token in CI secrets is one workflow misconfiguration away from being next week's item.


Two vocabularies, one control

Risky Business arrives at default-deny egress and per-task credentials from the escape side. Daniel Miessler's reporting arrives at the same place from the injection side: an agent's reading context must not hold send or write tools, because a payload only needs the ability to read untrusted content and trigger an outbound action inside one trust boundary to become self-propagating. Remove that adjacency and worm mechanics break regardless of payload sophistication. Both conclusions are capability separation, described in different vocabularies and reached from different threat models, which means the control is not an artifact of one attack class.

Detection-based defenses lose this race on economics. An attacker iterating with a capable model outpaces filter patching, and the same ledger reports ransomware operators claiming LLM-driven command-and-control and intrusion crews using coding agents to plan hands-on-keyboard work. The RaaS claim is single-source and hedged by its own author, so treat it as direction rather than measurement. The planning consequence survives either way: attacker tempo becomes a compute question, and the mean-time-to-detect budget shrinks against it.

An escaped agent holding a short-lived, task-scoped credential that cannot reach the metadata endpoint is an incident, because the credential expires with the task and the blast radius expires with it. A long-lived key does not expire, so it grants everything its policy allows for as long as the escape goes unnoticed. That is a breach.

What to do

  1. Rewrite the agent-sandbox threat model to assume escape this sprint: minimal base image with CUPS and unused usermode-network features removed, cloud metadata endpoint blocked, per-task short-lived credentials, default-deny egress.

  2. Inventory every MCP server reachable by internal agents this sprint, pin versions, allowlist by publisher, and remove marketplace servers with no active maintainer.

  3. Move npm and PyPI publishing to OIDC trusted publishing with provenance attestations this quarter and delete long-lived registry tokens from CI.

Adyen's Pod Timeouts Were Conntrack GC, Not Table Size

Three unrelated systems failed at a ceiling no dashboard displayed: flow entries reaped too slowly, a GPU fed one page at a time, and a rate limit configured once and never revisited.

Why the control plane cannot see this failure

Pod creation reads as an orchestration operation. The CNI ADD path is network-dependent. kubelet calls the CRI, the CRI invokes the CNI plugin, and most production CNIs then make control-plane or IPAM calls over TCP. When the kernel cannot insert a new conntrack entry it drops the SYN. The CNI retries, the retry drops, kubelet's timeout expires, and the pod sits in ContainerCreating. Nothing in that chain emits the word conntrack. The observable output is an opaque timeout.

Adyen's case turns on a distinction: table full and capacity exhausted are not the same condition. Expired entries are not reaped at the moment they expire. The kernel GC worker walks the table incrementally on a bounded time budget, so under high connection churn the table stays saturated with entries that are logically dead and structurally present. Two consequences outrank the fix itself. Raising nf_conntrack_max alone hands the collector more ground to cover per pass. And if nf_conntrack_buckets is not scaled alongside it (target roughly max/4), insertion failures are traded for long hash-chain lookup latency, the same pain in a slower, harder-to-attribute form. Scan cadence and per-pass budget are kernel-version dependent, so check the node image before copying anyone's sysctl gist. An eBPF datapath does not remove this cliff. It relocates it into a BPF map with its own sizing and eviction semantics.

The ground truth already exists in conntrack -S: insert_failed, drop, chaintoolong, plus the count-to-limit ratio. Almost no fleet exports them. That is why a named alert becomes a multi-hour diagnosis.


The same shape on the GPU side

Datalab's Marker v2 documents the serving equivalent. The default scaling move for document parsing is more workers, each loading its own model copy into VRAM and feeding it one page at a time. A GPU's speed is batching, so a private replica fed serially never forms a batch, and it idles while its worker does CPU-side file I/O and output assembly. You pay for N model copies and get roughly the throughput of one. Marker v2 inverts the layout: many thin CPU workers into one shared inference server, with the parent reading server capacity and splitting it across workers. Two metrics settle the question in an afternoon, mean effective batch size and GPU SM utilization. Batch size near 1 while several replicas run is the bug, and it bills monthly.

Two traps sit in the published numbers. The marketed 23.7 pages/sec is OCR-disabled on a B200. The accuracy-leading balanced mode is 2.9 on the same hardware. The default mode is also hardware-dependent: balanced on GPU, fast on CPU/MPS, where fast reads equations from the PDF text layer instead of the rendered page. Identical code therefore produces different extraction quality on a laptop and in production, and math degrades silently rather than erroring. In a retrieval ingestion tier that surfaces four weeks later as unexplained answer-quality complaints.

The third instance is a memory budget, not a throughput one. KV cache bytes per token equal 2 × layers × kv_heads × head_dim × dtype_bytes, which for a Llama-3-8B shape is 128 KiB per token. That is about 1 GiB for one 8K sequence and 64 GiB at 64 concurrent sessions. That number sets max concurrency, not FLOPs.


The common diagnostic

Uber's move from static per-client rate limits to closed-loop load management is the fourth instance of the same class. A limit encodes a capacity estimate from the moment it was set, while real capacity moves with data volume, cache hit ratio, and replica health. Set it low and headroom is stranded. Set it high and it does nothing during the overload it was bought for. Ship the adaptive limiter and priority tiers, with a static floor as a backstop so a misbehaving control loop cannot shed to zero.

Each of these outages was one exported counter away from being a named alert instead of a multi-hour diagnosis.

What to do

  1. Export nf_conntrack_count over the entries limit plus insert_failed and drop counters from every node this sprint, alerting at 60% utilization and on any sustained insert_failed.

  2. Measure mean effective batch size and SM utilization on every GPU service sitting behind CPU workers this sprint, then pin parsing or inference modes explicitly in configuration instead of inheriting the hardware-dependent default.

  3. Run a closed-loop concurrency limiter in shadow mode in front of your highest-contention datastore this quarter and diff what it would have shed against your last three overload incidents before enforcing.

The bottom line

Today's items share a finish line, and it is never the deploy. One ends at a rotated credential, one at an asset nobody inventoried, one at a stripped base image, one at a counter nobody exports. That breaks the working assumption that shipping a fixed version closes a problem, and it moves the cost onto inventory and revocation paths most teams have never rehearsed under time pressure. Prove two things about your highest-privilege automated actor this week: that you can revoke one of its capabilities without rotating everything around it, and that you can name every place it runs.