LLMs are making formal software verification feasible at a much larger scale

383 points · 235 comments on HN · read original →

Points and comments are a snapshot, not live.

Claude proved security properties for 123 open-source libraries at the assembly level.

Anthropic partnered with Theorem to use Claude for formal verification of RISC-V-compiled binaries from 123 OSS-Fuzz libraries. Claude produced 180 million lines of Rocq proofs showing no crashes on inputs up to 64 MB across 239 entry points, patching bugs it found along the way. The proofs convert existing fuzz tests into formal specs, covering memory-safety and other properties. Bugs discovered include a counter underflow requiring a 20 MB input and a rediscovery of CVE-2026-56132.

What commenters are saying

Commenters split on whether LLMs will hit a capability wall. One camp invoked the "convex hull hypothesis," arguing LLMs excel at interpolating within known data but struggle with true extrapolation and novel abstractions. Another countered that formal verification is a scale problem, not a reasoning one, and that improving proof modularity is a software engineering challenge. A lower-ranked comment suggested AI creativity in art is bound differently than in math or code.

A top comment noted that writing proofs is akin to writing code in a niche functional language, and that agents trained to write better proofs would generalize to better programming overall.