Tracking down the 16-year-old WAL-reset SQLite bug

1122 points · 212 comments on HN · read original →

Points and comments are a snapshot, not live.

Tailscale tracked a 16-year-old SQLite bug causing corruption after months of outages.

Tailscale suffered 19 SQLite corruption incidents over six months, causing control-plane downtime for tailnets on affected shards. Despite aggressive automated recovery, the root cause remained elusive until they hired SQLite's developers, who created a tracing tool (tmstmpvfs shim) to isolate a race condition in WAL checkpointing. The WAL-Reset bug occurs when a write happens at a specific time during a checkpoint, causing pages to be silently lost. The fix was released in SQLite 3.52.0.

What commenters are saying

Commenters praised Tailscale for funding open-source development via SQLite's professional support contract, with several noting CEO Avery Pennarun's long-term thinking. Some questioned the use of SQLite for multi-threaded access, but others defended it as designed for one-writer, many-readers. The article clarified that the bug only triggers with multiple connections, not single-writer use. Meta-discussion touched on Tailscale's open-source control-plane alternative Headscale, with one user noting missing features like App Connectors.