It's the memory latency, stupid

222 points · 73 comments on HN · read original →

Points and comments are a snapshot, not live.

Memory latency, not compute, is the true bottleneck for CPU-bound database workloads.

A modern CPU can process ~400-800 instructions per single DRAM access (50-100 ns). Naive hash table aggregation on 99M rows with 17M unique keys shows 50.1% of CPU slots stalled on memory, retiring only 16.1% of operations, and pulling 4.5 GB from DRAM. Radix partitioning into 4096 buckets shrinks working hash tables to fit in cache, reducing memory stalls to 23.5%, raising retiring to 32.2%, and cutting DRAM pulls to 1.5 GB. Query time drops from 3.6s to 2.7s on a single core. The author's engine achieves 1.85s on one core and 172 ms across 16 cores.

What commenters are saying

Commenters share war stories of cache-aware programming feeling like a lost art. Several note that younger engineers often lack basic understanding of kernel interfaces or microarchitecture. A senior engineer describes pushback against cache-friendly refactoring as "premature optimization" even with benchmark evidence. One commenter argues that "clean" code patterns (pointers everywhere) actually hurt maintainability compared to index+generation tricks from 2000s game dev. Two camps emerge: those lamenting homogenization around engines like Unreal, and those defending existing engines as good enough for making games.