Your Review Loop Is a False-Confidence Signal — Rebuild the Gate
Behind the 78% number: AI code is optimized to pass review, not survive production. It compiles, reads clean, and matches idiom, so reviewers rate it higher than human code. Meanwhile it reproduces vulnerable training-data patterns and hallucinates dependencies. Veracode pegs that at roughly 50% insecure output. The feedback loop inverted. The cheap early signal, review, lies. The expensive late signal, an incident, is the only honest one.
Where sources converge
New Relic, Veracode, and GitHub each point the same way. New Relic quantifies the incident gap. Veracode's dependency-hallucination vector is the one that keeps me up: a hallucinated package creates a namespace a malicious actor can register, and now it's a supply-chain hole. And GitHub's Copilot review agent got worse with more tools. Generic instructions made it browse the whole repo instead of the diff. Rewriting the prompt to "start from the diff, narrow with grep/glob, read exact evidence" cut cost 20% with no quality loss. Under-specified AI explores exhaustively and confidently reports noise.
The bifurcation
Human review keeps its value for design, mentorship, and architecture. It fails as bug-catching at AI-PR velocity. Split the roles. Humans review intent and structure. Machines enforce correctness via property-based tests, contract tests, and mandatory SAST/SCA that cannot be bypassed by human approval.
At AI-generated PR velocity, code review as a bug-catching mechanism is theater. The real gate runs in the pipeline.
Concretely: instrument AI-authored and human-authored code paths separately, then correlate incident rates. Verify resolved packages against known-good hashes so a hallucinated dependency can't resolve. Treat AI output as untrusted until validated.
What to do
Instrument AI-generated vs human-authored code paths in observability this sprint and track incident correlation separately to size your actual exposure
Add non-bypassable pre-merge gates (SAST/SCA + property/contract tests) that reject a PR on failed checks even with human approval, within the next quarter
Enable lockfile hash verification against known-good registries to neutralize hallucinated-dependency squatting