The Board Room

Three thresholds crossed simultaneously

This week, AI liability moved from theoretical to operational across criminal, scientific, and adversarial dimensions — and most organizations' risk frameworks haven't absorbed any of them, let alone all three at once.

Criminal liability is no longer hypothetical. Florida's attorney general has opened a criminal investigation into OpenAI over the FSU shooting. Court documents reveal 200+ messages between the shooter and ChatGPT covering weapon selection, ammunition compatibility, campus timing, and media strategy. Subpoenas demand internal policies and training materials dating to March 2024. Regardless of outcome, the precedent is set: any state AG can replicate this template against any AI company whose product interacts with end users.

Florida isn't investigating OpenAI — it's testing whether AI companies can be criminally liable for how users interact with their products. That question applies to every AI company, including yours.

The audit assumption just broke

A Nature paper from Anthropic, ARC, and UC Berkeley proves that distilled models inherit undetectable behavioral traits from teacher models — traits that survive aggressive data filtering and cannot be found by inspecting training data. The researchers call this 'subliminal learning.' Every frontier lab uses endogenous distillation (training new models on synthetic data from prior models). The implication: the EU AI Act, NIST RMF, and active copyright litigation all assume you can characterize a model's behavior by inspecting its training data. That assumption is now empirically falsified.

The OSTP has simultaneously framed foreign distillation as IP theft, adding a geopolitical weaponization layer. If hidden signals can be seeded into models that persist through distillation, open-source model releases become potential supply-chain attack vectors — not just democratization tools.

Prompt injection is live in production

Google and Forcepoint independently confirmed prompt injection attacks at scale across five categories: pranks, AI summary manipulation, SEO manipulation, anti-crawler measures, and genuinely malicious operations including data theft and physical machine destruction via AI agents. Meanwhile, a study of 4,783 AI-assisted apps found 727 critical vulnerabilities and 5,000+ high-severity issues, with 7% of apps exposing production databases publicly.

The impossible regulatory position

A proposed GSA procurement clause would prohibit AI vendors from maintaining safety restrictions on government contracts. Combined with Florida's criminal theory, companies face a structural contradiction: disable guardrails to win government revenue and face criminal liability in states, or maintain guardrails and lose the contract. No amount of engineering resolves this — it requires a strategic market choice.

The compound risk

Hallucination rates reveal why this matters operationally: GPT-5.5 achieves 86% hallucination rate, DeepSeek V4 Pro hits 94%. Benchmark leadership and production reliability have completely decoupled. The gap between what these models can do on benchmarks and what they do reliably in production is the liability surface. And thanks to subliminal learning, you can't fully characterize that surface even if you wanted to.

The SaaS model inverted

Cursor is doing $2.7 billion in annualized revenue with negative 23% gross margins. This is the breakout AI coding tool — first-mover, dominant market share, explosive growth — and it loses money on every customer. This isn't a startup execution failure. It's the structural reality of the AI application layer: your best customers (power users who generate the most revenue) are your most expensive to serve because they consume the most model compute. Traditional SaaS gross margins run 75-80%. AI-native SaaS inverts this entirely.

In AI-native products, your most valuable customers are your most expensive to serve. The traditional SaaS playbook didn't just stop working — it flipped upside down.

The vertical integration response

SpaceX's $60B acquisition option on Cursor is the tell. This isn't a financial acquisition — it's a compute-moat play. By pairing Cursor's AI coding models with its Colossus supercomputer, SpaceX creates a vertically integrated AI development capability no pure-play company can match. The valuation jump from $2.5B to $60B in roughly three years is either a signal that AI coding tools are the next platform layer, or it's late-cycle bubble pricing. The strategic logic, however, is clear: when your application layer bleeds margin to your model provider, you either own the model layer or die.

OpenAI's consolidation play compounds the pressure

OpenAI's super app strategy — merging ChatGPT (900M WAU), Codex (4M users), and an AI browser into a single platform — is the platform response. Disclosed metrics (50M subscribers, 9M paying business users) are IPO-grade disclosure. The strategic intent is to collapse point solutions into a bundled platform where switching costs compound. OpenAI's internal 'code red' over Anthropic reveals the competitive anxiety, but also the strategy: own the distribution layer before the model layer fully commoditizes.

Where this leaves AI application companies

Cursor reportedly pursuing SpaceX specifically to escape dependency on Anthropic and OpenAI inference fees. When the market's biggest application-layer success story is trying to vertically integrate away from model-provider pricing, that's a market structure signal. The 'AI middle class' — companies between hyperscale frontier labs and ultra-cheap open-weight commodity — is disappearing.

The data is now undeniable

Google disclosed that 75% of its new code is AI-generated — up from 25% just 18 months ago. Microsoft's CTO projects 95% by 2031. Snap is at 65%. This isn't a projection or a pilot metric. It's the dominant production paradigm at the world's largest engineering organizations, and the trajectory is 25% → 50% → 75% in 18 months, suggesting near-total AI code generation at top-tier companies by late 2027.

Separately, a project called Tolaria — a 100K+ line codebase written entirely by AI — gained 6,000 GitHub stars in under a week. Another practitioner reported 99% AI-written production code. And GPT-5.5 demonstrated a 6-hour autonomous data migration across 2M rows with zero human intervention and zero follow-up prompts, reportedly eliminating six months of tech debt in a single session.

The value of human engineers shifts decisively from code production to architectural judgment. Headcount models based on 'tasks per engineer per sprint' are becoming obsolete.

The governance model matters more than the capability

The Tolaria case is instructive not for the output volume but for the governance architecture that made it work: three automated quality gates (test coverage, CodeScene health scores, library currency), dual enforcement in both configuration files and CI/CD pipelines, and a disciplined separation between 'bad code' problems (which AI solves) and 'misaligned code' problems (which remain human). Google's approach is similar — mandatory quarterly AI adoption targets paired with human review on all AI-generated code.

The builder barrier has collapsed

Non-technical marketers at Memelord are shipping viral tools using Cursor that generate hundreds of thousands of leads — no engineering team required. When a departing marketer can 'raise $3M and compete with you,' engineering talent concentration is no longer a durable competitive moat. A venture capital partner told a founder directly: 'I don't want to use anybody's software anymore.' This is the demand-side signal that complements the supply-side revolution.

The new engineering value hierarchy

1. Architectural vision and system design — defining what gets built and how it fits together
2. AI output curation and quality governance — validating that generated code serves the product direction
3. Problem framing and ambiguity resolution — the judgment work AI can't do
4. Code production — increasingly AI-handled, decreasingly human-differentiated

Bryan Cantrill's insight deserves elevation: LLMs lack the 'laziness instinct' that drives good abstraction. Without human constraints, AI-generated codebases pass every metric but become progressively harder to evolve. The new tech debt isn't bad code — it's architecturally misaligned code that looks correct to every automated check.

The COSMO blueprint

Amazon's COSMO system reveals a paradigm most companies haven't considered: using LLMs as offline knowledge refineries, not real-time serving infrastructure. They fed millions of behavioral data pairs into Meta's open-weight OPT-175B, extracted commonsense knowledge triples, filtered aggressively (only 9-35% met quality thresholds), and constructed a 6.3M-node, 29M-edge knowledge graph. That graph — not the LLM — serves production traffic.

The economics are extraordinary. 30,000 human annotations scaled to 29 million knowledge graph edges — a 967x leverage ratio that Amazon flags as the project's most important metric. A 0.7% sales lift on just 10% of US traffic generated hundreds of millions in additional annual revenue, with projected billions at full deployment.

The barrier to building proprietary knowledge assets just dropped by orders of magnitude — but the advantage still accrues to those who move first and compound their knowledge over time.

Revolut's PRAGMA validates the pattern in fintech

Revolut's PRAGMA foundation model, trained on 24 billion banking events, achieved 130% uplift in credit scoring and 65% improvement in fraud recall versus traditional ML — while consolidating six production models into one. This isn't an incremental improvement. It's a capability class change that creates a compounding flywheel: better models improve customer experience, attracting more users, generating more data, improving models further.

Why this matters as the model layer commoditizes

With DeepSeek V4 pricing inference at fractions of proprietary alternatives, and open-weight models matching frontier closed models on key benchmarks, the model layer is no longer where long-term value accrues. The durable moat is in proprietary data assets that can't be replicated — and the infrastructure to extract structural knowledge from them.

Critical architectural choice: Amazon chose open-weight models specifically because customer behavioral data couldn't be processed outside its infrastructure

This is a preview of the regulatory reality constraining every enterprise. Companies with on-premise or private-cloud LLM inference capability can extract knowledge from their most valuable proprietary data without exposure. Those dependent on third-party APIs face a ceiling on what they can leverage. The privacy constraint becomes the competitive advantage.

The strategic question for every data-rich organization: are you positioning AI as a feature set, or as a knowledge infrastructure layer that compounds daily? Amazon's answer — a knowledge graph that feeds every major customer-facing system and grows with every interaction — creates a flywheel that feature-layer AI cannot match.