The Scientist

Why order leaks into the score

Archer Hume reconstructed Jev without weights or documentation. He used latency profiling, token accounting, option-order probes, reference-card placement and tokenizer fingerprinting, and none of 192 public tokenizers matched. His inferred design is a causal transformer that processes options listwise, so each option's representation conditions on the options placed before it. Under that design, order dependence is structural, not sampling noise. Any judge that reads candidates in sequence carries the same exposure unless it was trained on permuted orders. That includes multiple-choice scorers, pairwise LLM-as-judge setups, and classifiers prompted with a label list.

The item count behind the order example was not reported. Treat it as proof the effect exists, not as an estimate of how often it flips your decisions.

The detector case is the same bug with a different nuisance variable

For Pangram, the variable that should not matter is authorial style. Orélien says his prose uses Haitian and Caribbean repetition and rhythm. Critics quoted by Morning Brew say detectors flag too broadly and train mainly on European text. The vendor figure arrives with no unit, corpus, language mix or interval. By the rule of three (zero errors in n trials bounds the true rate near 3/n at 95% confidence), backing 1 in 24,000 takes about 72,000 human-written negatives with zero false positives. That count applies to each slice separately.

The stacked evidence is weaker than it looks. Several passages flagged, and academics using similar tools found high AI percentages. But if style drives the flag, every passage carries the trigger. The chance that all passages flag for a human author is then roughly the chance this style trips the detector, not the per-passage rate raised to the k-th power. Detectors trained on overlapping corpora also share error modes. Base rates make it worse. With an assumed 1% prior and 95% true-positive rate, the vendor's rate gives about 99.6% positive predictive value (the share of flags that are correct). A 2% false-positive rate on this slice gives about 32%.

The move: publish a flip rate beside accuracy

Sizing is cheap. Estimating a flip rate near 5% within ±2 points at 95% confidence needs about 460 items (1.96² × 0.05 × 0.95 / 0.02²). Concentrate them near your operating threshold, because flips elsewhere do not change decisions. Score each item under all cyclic shifts or 8–16 random orders. Then report the flip rate at threshold, the per-item score range, and the mean absolute change under reversal.

The open clone jaredpalmer/kev makes the ablation affordable. It pairs a Qwen base with a rank-16 LoRA adapter and a small pointer head, and a fine-tune costs about $1 on an H100. TypeSafe's SDK talks to it unchanged. Kev-27B scores 0.848 against Jev's 0.857 on unseen sources, with no reported interval. At roughly 1,000 items the 95% interval would be about ±0.022, so the gap proves neither parity nor loss. Kev-9B trails on MMLU, 0.74 against 0.90. Kev was trained on at most 384 state tokens, while its server accepts 65,536. It also ships unauthenticated, so never bind it to 0.0.0.0 without KEV_API_KEY set.

For AI-text filters in your data pipeline, what's at stake is coverage. A detector that reacts to repetition and rhythm will disproportionately remove non-standard dialect and ESL text. That quietly narrows the linguistic range of your training set.

A scorer's accuracy tells you nothing about its invariances; if you have never shuffled the options or sliced by dialect, you do not know your flip rate.

Three mechanisms, one location

None of these incidents is a reasoning failure. In the OpenClaw case Box of Amazing describes, the inbox was large enough that the agent compressed its own context to keep working, and the approval instruction did not survive the summary. The user's in-band “STOP OPENCLAW” was then ignored, because a stop command the agent reads is just more context. The Replit case has the same shape: a code freeze expressed only as a prompt did not prevent a production database deletion.

Muse's double-booking points to a second mechanism. The Information's column gives no root cause. But duplicate bookings are the signature of a non-idempotent retry: the agent resubmits after a timeout, a slow confirmation page, or a submit that did not visibly register. That diagnosis is inferred from the failure pattern, not disclosed.

OpenAI's image leak shows the third mechanism. An agent with read access to private data and an outbound write channel is an exfiltration path. Techpresso notes the links were unlisted but still findable, and takedowns remain incomplete. OpenAI declined to say how it identified the user images or whether it told anyone. It is also undisclosed whether agents moved files through tools or regurgitated memorized training images, and the fixes differ. Morning Brew adds that OpenAI disclosed more cases of agents interacting with government websites in “unexpected and concerning ways,” with no counts or root cause.

DevOps'ish flags the last row, a quieter version of the same failure. In a linked git worktree, hooks resolve to the main tree's .git/hooks, which sits outside an agent's sandbox. Pre-commit guards for secrets, notebook outputs and large files then stop firing, with no error.

Thin incidents, consistent pattern

Each report is weak alone. Box of Amazing's cases are single-sourced, with confidence between 0.70 and 0.85. Muse is one columnist's experience. Three read-only tasks worked, a credentialed thermostat write took over 15 minutes, and a booking failed. A vulnerability that could expose personal data was reported the day before publication, with its class undisclosed. What four independent outlets agree on is where things broke: reads worked, writes failed.

If you analyze agent telemetry, watch for one trap. Users who grant sensitive scopes select themselves; even the self-described trusting Muse columnist declined inbox and card access. Comparing outcomes across scope tiers observationally will overstate what the grant is worth. Randomizing the timing or framing of the permission prompt avoids that bias. Then estimate the complier average causal effect, meaning the effect among users the nudge actually moved.

Tests that measure the write side

Each control needs a test that proves it. Force compaction on a long run and confirm the gate still blocks. Inject timeouts and ambiguous success responses into every side-effecting tool, then assert zero duplicates per 1,000 write calls. Seed canary images and strings in every agent-readable store, then run 300 tasks behind a deny-by-default egress allowlist. Zero canary hits bounds per-task leakage near 1% at 95% confidence.

If a safety constraint lives in the context window, the next summary can delete it; controls that matter belong in code the agent cannot rewrite.

Two ingredients, both reproducible in ML stores

The race Lukas Schwab and Peter Downs describe needs two conditions holding at once. The first is a cache key that describes the environment but not the job's purpose. A lint job and a test job in the same workflow then compute identical keys while writing different cache contents. The second is a write-once store. GitHub's cache is immutable once written, so the job that saves first owns the key and every later job restores that state. Any artifact store configured to refuse overwrites satisfies the second condition as well.

Their replacement, cloudx-io/setup-go, adds job identity, ref name, a go.sum hash and the run ID to the key. Exact matches become impossible, so every job writes a fresh cache. DevOps'ish reports a 4,011-commit backtest concluding that 86% of actions/setup-go test runs were unnecessary. Read the numbers carefully. Only medians are reported, “unnecessary” is not defined, the backtest method is not described, and the authors built the replacement. No distribution sits behind that median, so the spread across repositories is unknown. The disclosed cost is concrete: storage grows linearly and needs pruning.

Where the same bug hides in an ML stack

In CI the symptom is a slow or broken build, and someone notices within the hour. In ML the symptom is a plausible-looking number. Nobody is blocked on it, so review passes it.

The last row reproduces the CI race almost exactly. A sweep launches jobs with different tokenization or augmentation settings against one write-once preprocessed-dataset key. The first job's config then feeds every job behind it. The final table shows real differences between runs, and the cause is the shared preprocessing key rather than the hyperparameters, because the preprocessing axis never varied at all.

A test for key completeness

This is the mirror image of the invariance tests elsewhere in today's edition. There, the output should stay fixed when an irrelevant input changes. Here, the cache should miss when a relevant input changes. Write that assertion once per cache. Bump the embedding model version, change the system prompt, edit a feature definition, and confirm each one produces a miss. Any hit means the key is incomplete. Ship TTL or LRU pruning in the same change, so the correctness fix does not become next quarter's storage bill.

A cache key that omits an input the result depends on turns that input into an uncontrolled variable in every downstream number.