Engineering & Technical

The Engineer

The Signal

A 149 GB Snowflake query landed in your cost dashboard as -1.5 GB, exit code zero.

The Parquet unload declares nine QUERY_HISTORY columns as INT32. Anything past 2,147,483,647 wraps silently, deterministically, on exactly the rows with the largest scans. So the chargeback model and the optimization backlog you built on those exports are wrong for precisely the queries worth optimizing.

In Play

  1. Warehouse Telemetry Is Returning Negative Bytes

    Espresso AI documented Snowflake reporting a 149 GB query as having scanned -1.5 GB, per TLDR Data's roundup. Unloading QUERY_HISTORY to Parquet writes large integers as INT32, and nine columns overflow. Every chargeback model and query-optimization backlog built on exported warehouse telemetry is wrong exactly where your spend is largest. The same shape appears in Tailscale's 16-year-old SQLite WAL checkpoint race: both failures live off the common path, not on it.

    Ask Clarity
    Try
  2. Akira Reboots Windows To Unload Your EDR

    Akira ransomware now flips the Windows BCD safeboot flag and reboots hosts into Safe Mode, where most third-party endpoint agents are never started by the operating system, per CSO First Look. Tamper protection and behavioral detection do not fail; they never run. The same minimal environment broke Akira's own encryption routine, so some victims get corrupted data with no decryptor available at any price. Backup integrity is now the entire recovery story.

    Ask Clarity
    Try
  3. DeepSeek Gave Away The Harness, Repriced The Tokens

    DeepSeek published Harness — its full agent runtime of tools, memory, execution loop, sandbox, and scheduling — under an MIT license, per Turing Post. In the same week it raised V4 API prices by 50% to 1,100% and introduced peak-rate pricing. If your roadmap lists the agent harness as a moat, that layer now ships with a one-command getting-started path. Any hardcoded dollars-per-million-token constant in your cost model is also wrong now, because per-request price depends on time of day.

    Ask Clarity
    Try
  4. Agent Session Cost Grows With The Square Of Turns

    Claude Code appends every file read and command output to the conversation and re-sends the whole thing on the next turn, which makes a session's input bill roughly quadratic in turn count. Prompt caching prices that prefix at one-tenth of input, per the token breakdown in Simplifying AI, but it divides the constant rather than changing the curve. Caching is also prefix-exact: one rotating timestamp near the head of your prompt and you pay full price for the entire history.

    Ask Clarity
    Try
  5. Ampere Silicon Is Contracted Through 2029

    CoreWeave disclosed a renewed contract for 2020-era A100s running through 2029 at what CEO Mike Intrator called full-freight pricing, per TLDR Hardware. That implies nine years of economic life on silicon five architecture generations old, so sm_80 stays in the fleet you schedule against. Delete the GPU-hour price-decay line from your TCO model and replace it with a capacity-availability constraint. Any serving kernel that hard-depends on FP8 or sm_90 features strands that capacity.

    Ask Clarity
    Try

Deep Dives

The Unload Path Nobody Tests Still Exits Zero

Cost dashboards, an embedded database, and CDC sinks each broke in a mode their vendor never exercised, and every one of them returned a clean exit code while doing it.

Why the number goes negative

A signed 32-bit integer stops at 2,147,483,647. A 149 GB scan is roughly 160 billion bytes. Write that into an INT32 column and it wraps to a small negative value, which is the -1.5 GB Espresso AI reported. Nothing errored. The unload wrote a file, the job went green, and the corruption is deterministic. It lands on exactly the rows with the largest values.

Trace the propagation before ranking the urgency. Sum bytes scanned per team and the heaviest queries subtract from the total, so chargeback under-bills the biggest consumer. Sort an optimization backlog descending by bytes scanned and the worst offenders sink to the bottom. Join exported telemetry against a job registry on any of the nine wrapped columns and rows silently fail to match. The remedy is two lines of SQL plus one policy: cast the affected columns before export, then add a schema assertion that fails the pipeline on negative bytes, negative durations, or negative row counts. Platform telemetry is data nobody on the team wrote and nobody tests. Validate it at the boundary like any third-party payload.


The pragma you tuned is code upstream never ran

Tailscale spent six months tracing repeated database corruption to a race between SQLite's WAL checkpointing and write transactions, a bug that sat latent for 16 years. It surfaced for them because they checkpoint aggressively and manually. That is the transferable part. The default path collected 16 years of accidental fuzzing from millions of deployments. A configuration hand-tuned for throughput gets whatever coverage the local suite gives it, which is usually none. Good engineering, wrong test population.

So the work is an inventory, not a patch: enumerate every embedded SQLite deployment, write a one-line justification for each non-default pragma, pin a post-fix version, and put PRAGMA integrity_check into a startup or health check so corruption arrives as an alert instead of a support ticket.


CDC gives you ordering, not idempotency

Change data capture buys retryable, ordered, observable downstream writes. It does not make a destination safe to receive the same event twice, and replay after recovery is precisely when it will. Classify every sink into one of three buckets:

  1. Idempotent by upsert. The write is naturally convergent.
  2. Idempotent by key. A durable dedupe key, such as source LSN plus primary key, persisted atomically with the side effect.
  3. Explicitly at-least-once tolerant. Double-counting is documented as acceptable by an owner.

Anything unclassified is a duplicate-write bug waiting for the next replay, when nobody is reading the diff carefully. Metrics double, side effects fire twice, and the incident gets blamed on the replay rather than the sink.


The same discipline applies to numbers you did not produce

There is a tension inside the data reporting. The same material arguing that a valid benchmark must disclose workload shape, cache state, and scaling limits also carries a 3.4x DuckDB-versus-EMR-Serverless claim measured on 1,000 JSON files, with no cache or cold-start disclosure. At that size the ratio is mostly cold-start tax and single-threaded JSON parsing. The direction is probably right: single-node wins the sub-scale tier. The ratio does not transfer.

The security column of this briefing rhymes exactly. CSO First Look describes a defense that never starts rather than one that fails. Same reflex in both columns: assert the invariant on the path you rarely take.

Every correctness failure here came from running a trusted system slightly off its happy path, which is also where the untested code lives.

What to do

  1. Grep every pipeline for QUERY_HISTORY reads today, add explicit casts on the nine overflowing columns before Parquet unload, and invalidate the exports already landed in the warehouse.

  2. Add a schema assertion this sprint to every telemetry pipeline that fails the run on negative bytes, durations, or row counts.

  3. Classify every CDC sink as idempotent-by-upsert, idempotent-by-key, or explicitly at-least-once-tolerant by the end of this sprint, and file the unclassified ones as bugs.

Akira Sets A Boot Flag And Your EDR Never Starts

Endpoint defense assumes a runtime; a Safe Mode reboot removes it, and this crew's broken encryptor means your backups are now the only recovery path that exists.

The registry key that decides whether the agent runs

Safe Mode boots a minimal service set. The list lives under HKLM\SYSTEM\CurrentControlSet\Control\SafeBoot\Minimal (or \Network). Most third-party endpoint agents never registered there. So the chain is dull: set the BCD safeboot flag, optionally add the payload's own service under the SafeBoot key so it does start, force a reboot. On the far side, tamper protection, behavioral analytics, and agent self-defense are not defeated. They are runtime constructs, and there is no runtime.

Categorization drives the response. This is a control-plane attack, the same species as deleting a cloud audit trail before acting rather than evading its detection rules. The conditions under which the logic executes get removed. Snatch and several ransomware-as-a-service families did this years ago, per CSO First Look. Continued success says more about default endpoint posture than about Akira's sophistication.


The encryptor bug removes the worst option

The minimal environment broke Akira's own encryption routine. Some victims got corrupted rather than encrypted data. That is bad news. Paying for a decryptor was already a poor option; now it is not an option. Assume the next build fixes the encryptor, which puts the defender window at weeks, not quarters.

The honest test is a restore drill run on the assumption that the data is unrecoverable in place and no decryptor will ever exist. Take the largest stateful service, restore from immutable off-domain backups, measure actual RTO against the runbook number. The gap usually shows up in restore-and-verify, not in the copy.


The management plane is the force multiplier

The Hacker News reports a patched VMware vCenter flaw chained by a suspected China-nexus actor into Babuk-derived ransomware, exploited within days of the fix landing. Two engineering consequences follow. First, the hypervisor management plane is tier-0 infrastructure: dedicated management network, MFA on an identity independent of production SSO, immutable off-domain backups, timed restore drill. One control point encrypts the entire VM estate.

Second, the payload no longer identifies who is at the keyboard. A suspected state-nexus actor shipping commodity ransomware means motive cannot be inferred from encryption. Branch into exfiltration hunting on every encryption event, before the restore destroys the evidence.

Where the reporting is thin, say so. The vCenter and macOS Screen Sharing items arrive with no CVE identifiers and no affected version ranges, so scope exposure from vendor advisories and the national CSIRT feed rather than from summaries. The Akira mechanism is specific enough to build detections against.


Detections worth writing this sprint

  1. Alert on writes to the BCD store and to the SafeBoot service keys. Legitimate changes are rare and operator-initiated, so the false-positive budget is small.
  2. Correlate forced reboot plus agent heartbeat loss into one high-severity signal. Endpoint telemetry is gone across the boot, so identity, DHCP, and DNS re-registration events carry the detection.
  3. Test one nuance in the local image: BCD modification can perturb measured-boot PCRs and trigger a BitLocker recovery prompt, which breaks unattended attacker automation. A genuine speed bump with real support load. Verify it before claiming it as a control.
Akira does not bypass your EDR. A boot flag decided the agent would never start, before any detection logic existed.

What to do

  1. Get a written answer from your endpoint vendor this week on whether the agent service registers to start under minimal Safe Mode.

  2. Ship a SIEM rule this sprint that fires on BCD safeboot writes or SafeBoot registry mutation correlated with agent heartbeat loss.

  3. Run a restore drill this sprint on your largest stateful service assuming corrupted data and no decryptor, and record measured RTO against the documented number.

The Free Agent Runtime Stops At Irreversible Tool Calls

An MIT-licensed runtime formalizes rollback for four classes of side effect and none for the fifth, which is exactly the class your agent incidents come from.

What Cordis actually guarantees, and under which assumptions

Harness runs on Cordis, a runtime with two invariants. Effects reversible: a component records how its changes get withdrawn. Dependencies reactive: dependents deactivate before a service they need is removed. Under reversibility, declared dependencies, and independent effects, the paper claims a mutated system reaches the same observable state as one built from scratch in that configuration. Saga pattern plus topologically ordered teardown, moved out of application code and into the runtime. Real tools violate all three assumptions.

Provenance persuades more than the theorem. Cordis came out of Koishi, a four-year-old chatbot framework with thousands of community plugins running on QQ, Telegram, and Discord; its creator now works at DeepSeek, per Turing Post. That is plugin lifecycle hardened by an untrusted third-party ecosystem, which is the code most in-house agent frameworks never had to write.

Creator mode: lacking a capability, the agent writes a temporary plugin, asks permission to load it, uses it, removes it. Codegen plus dynamic loading inside the process boundary. Unloading does not undo what the plugin did. Read the sandbox implementation locally, away from production credentials.

Classify effects before trusting any rollback story

Effect classExampleReversal mechanismSafe in an autonomous loop?
Pure / read-onlySearch, retrieval, static analysisNone neededYes
Local mutableScratch filesystem, temp pluginSnapshot and discardYes, in a sandbox
Idempotent external writeUpsert by key, tagged config writeCompensating upsertYes, with an audit log
Non-idempotent external writeTicket creation, row insertBest-effort compensating deleteOnly with budget and rate limits
IrreversibleEmail send, payment, DNS cutover, hard deleteNoneNo — two-phase plus human gate

Cordis buys rows one through four. Row five is a product decision no formalism touches, and row five is where agent incidents originate. Cheap port: three fields per tool registration, side-effect class, compensating action or an explicit irreversible flag, declared dependencies, plus enforced deactivation order on unload. Days of work, and half-applied multi-step trajectories become something a chaos test can exercise by yanking a dependency under load.


Two other findings point at the same layer

Import AI reports Inherent's Faraday, a 27B model post-trained on Qwen-3.6-27B that drives OpenAI Codex as a tool, reportedly beating vanilla Opus 4.8 and GPT-5.5 on 73% of in-distribution ML tasks and 60% of held-out science tasks. Self-reported: rubrics generated by Claude Opus 4.7, scoring by a Codex-based judge, so training signal and eval share a topology. Clone that stack and the policy learns to satisfy rubric text. DiG-bench, same reporting, is cleaner. Several frontier models cleared Tier 6 only when given a harness, so the leaderboard ranks model-plus-scaffold.

Lenny's Newsletter, third angle. Codex could not emit a valid CAD file; Codex driving CLO's interface could. Orchestrator plus deterministic executor, and the driven surface sets the operational properties. GUI driving has no idempotency, no transactional semantics, no error taxonomy, so a failed step and a slow step look identical from outside.

DeepSeek's own behavior is the disagreement worth noticing. This material says orchestration is where value accrues; DeepSeek published the orchestration layer and raised the price of inference. Both hold if the durable asset is the contracts rather than harness code: effect classification, pinned scaffold versions, a judge validated against human grading.

The harness is free. The reversibility contract for in-house tools is not downloadable, and neither is the eval that proves it holds.

What to do

  1. Add three required fields to every tool registration this sprint — side-effect class, compensating action or explicit irreversible flag, declared dependencies — and enforce deactivation ordering on unload.

  2. Gate every irreversible tool call behind proposal, policy or human approval, then execute this sprint, logging the intent before the call fires.

  3. Make harness and scaffold version a required pinned field in every model-eval config this quarter, and mark last quarter's model comparisons untrusted until they are re-run.

The bottom line

The pattern under these items is uniform: each failure happened in a mode the vendor shipped but never exercised — an export, a hand-tuned durability setting, a minimal boot, a recovery replay — and every one of them returned something that looked like success. That breaks the reflex that a zero exit code, a host that came back up, or a downloaded archive is evidence the invariant held. Your assertions have to live on the paths you only reach during recovery, migration, and teardown. Pick the one off-nominal path your system takes only when something has already gone wrong, and make it fail loudly this week.