The Watch

What Happened

Attackers socially engineered Meta's AI chatbot into changing the registered email address on high-profile Instagram accounts. The mechanism: a credential-reset path fronted by an LLM that treated the interaction as a support conversation rather than an identity-proofing event. The model did not breach anything. It was talked into doing so on the attacker's behalf. In MITRE ATT&CK terms, this maps to Account Manipulation (T1098) and Account Access Removal (T1531), executed through a non-human intermediary.

The attacker did not bypass authentication. The attacker convinced the AI to bypass it for them. That is a different class of problem than anything in the CVE database.

Why This Generalizes

The pattern — convince the AI to perform a privileged identity action that a human agent would have flagged — applies to any LLM wired to tools that mutate state. Three conditions are sufficient:

1. The LLM has access to identity-state-changing APIs (email change, MFA reset, password recovery)
2. The LLM accepts conversational input from an untrusted party
3. There is no out-of-band verification gate between the LLM's decision and the action

Multiple sources confirm this is the default architecture across most consumer AI features shipped in the last 18 months. The Instagram incident is the first clean public demonstration. It is not the only vulnerable surface.

OpenAI's Response: Lockdown Mode

OpenAI shipped Lockdown Mode this week. It is the first vendor-togglable prompt-injection mitigation on a major LLM, and it works by disabling capabilities entirely: Deep Research off, Agent Mode off, internet image fetching off, file downloads off. Read it as an admission that prompt injection has no clean technical fix. Amputation, not a cure.

It is available on all personal accounts including the free tier. Enterprise/Team tenants are not explicitly covered in the rollout, which means DLP and tenant-level policy stay load-bearing for business users.

Cross-Source Pattern

Three sources converge on the same structural finding from different angles:

• Source A documents the Instagram exploit and maps it to MITRE ATLAS (LLM Prompt Injection → Privilege Escalation via Tool Use)
• Source B frames Lockdown Mode as the vendor response, confirming the industry treats this as architecturally broken
• Source C notes Microsoft expanded its AI agent failure-mode taxonomy by 7 new categories, meaning agentic deployments shipped in the last 12 months were threat-modeled against an incomplete framework

Worth flagging: Meta has not described patch status or scope. OpenAI's mitigation disables the features rather than securing them. No vendor has demonstrated a solution that preserves capability while preventing the attack.

The Headline Vulnerability

The bug is a remote code execution vulnerability in Hugging Face Transformers, triggered by crafted model configuration files. The payload rides in on something most pipelines treat as inert metadata. Successful exploitation lands on GPU-accelerated inference hosts, historically the worst-instrumented boxes in the enterprise. The package has 2.2 billion installs.

That 2.2 billion is the ceiling, not the exposure. Real exposure depends on how many installs are reachable by an attacker who can stage a malicious artifact, versus pinned, air-gapped, or otherwise inert. For most defenders the distinction is academic. The reachable population is still large.

The model config was not supposed to be executable. The MCP server was not supposed to read ~/.aws/credentials. Each violates a developer mental-model assumption. That is why they work.

The MCP Problem

Claude Code's Model Context Protocol integration layer has known weaknesses, and developers are actively wiring themselves into them. MCP is the protocol teams use to hand an LLM tools, files, and credentials. A client vulnerability is a vulnerability in everything the client was trusted to touch: source, secrets, local filesystem, cloud creds.

Claude Code ships with seven permission modes. Two of them, bypassPermissions and dontAsk, suppress interactive approval on tool calls and shell commands. A developer running either mode on a host with production credentials has delegated execution authority to a model whose input channel includes every README, issue, and dependency in the repo.

Detection Gap Analysis

The three sources converge on one pattern: the AI stack was assembled quickly, by application teams, on top of libraries that were research code in 2022. It is production attack surface now. Detection engineering is 12 to 18 months behind it.

Microsoft's Expanded Taxonomy

Microsoft this week added seven new attack categories to its AI agent failure-mode taxonomy, including prompt injection and tool abuse variants. Any agent deployed in the last 12 months was threat-modeled against an incomplete framework. Re-threat-modeling against the updated taxonomy is a deliverable now, not a discussion topic.

The Numbers

GitHub disclosed 17 million agent-generated pull requests in March 2026. One month. One platform. The same merge pipeline humans use. On June 1, Copilot switched to usage-based billing with semantic routing across model tiers. GitHub also shipped Chronicle, which persists agent sessions — prompts, generated code, debug context — to GitHub-managed cloud for queryability.

None of this shipped as a security announcement. Read together, three control planes change at once.

Code review was designed around the assumption that the author was a human with a job title and a Slack handle. Agents do not have those. They have API keys and prompts, and the prompts are increasingly authored by other agents.

Three New Attack Surfaces

1. Agent-Authored Code at Unreviewable Volume

AppSec controls built around pull-request review assume a human on at least one end of the diff. At 17M PRs/month, that assumption is a budgeting problem, not a policy. Known failure modes: slopsquatting via hallucinated dependencies, over-permissive IAM in IaC, secrets in test fixtures, license contamination. SAST and SCA tools tuned for human-authored cadence will not keep pace without re-architecture.

2. Financial Attack Surface via Billing

Under usage-based Copilot, a compromised developer credential no longer just leaks code. It spends money. A stolen PAT looped against an agent endpoint, with the semantic router free to escalate to Opus or GPT-tier models, is a financial denial-of-service with an invoice attached. Most SOCs do not alert on Copilot spend anomalies today.

3. Chronicle as Ungoverned Data Sink

Chronicle persists prompts, generated code, and debug context outside the customer tenancy. Regulated data — PII, PHI, source IP — crossing that boundary without DPA coverage is a GDPR and SOC 2 finding waiting to be written up. DLP policies do not yet inspect IDE-to-Copilot-to-Chronicle traffic.

Concentration Risk

Publicly: GitHub infrastructure is shedding load into Azure after hitting single-data-center limits. Rumored, but consistent with user reports: West Coast network paths saturating under agent traffic. Dev pipeline (GitHub), identity (Entra), productivity (M365), and AI tooling (Copilot/OpenAI) now share fate. BCP plans that model a GitHub-only outage are stale. Correlated degradation across the stack is the architecture, not a tail risk.

GitHub's VP Engineering has stated the API layer is evolving to be agent-centric. New auth flows, new scopes, new machine identities. Most IAM inventories have catalogued none of them. Non-human identity sprawl at GitHub is the next inventory problem, and it is already in production.