Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache
Points and comments are a snapshot, not live.
Cloudflare cut 100 TB of memory from its 1.1.1.1 DNS cache with five structural optimizations.
Cloudflare's Big Pineapple DNS cache holds over 250 billion entries. Engineers replaced Vec and String fields with Box<[T]> and Box<str> to eliminate capacity overhead, combined three record lists into one with u16 offsets, dropped redundant owner names for most records, boxed large enum variants, and finally stored record data as raw bytes in a single buffer. The changes freed roughly 100 TB of memory fleet-wide, cut lookup latency by 19%, and raised insert throughput by 43%.
What commenters are saying
Many commenters questioned why trivial optimizations like swapping Vec for Box<[T]> weren't caught in initial design reviews. Defenders cited Rob Pike's rules: don't optimize until measurement proves a bottleneck, and early focus on correctness and time-to-market outweighed memory savings. A few noted that the choice of Rust made advanced tricks (e.g., variable-length entries alongside struct fields) harder than in C or Zig. Others pointed out that the 100 TB freed is equivalent to roughly 130 servers' worth of RAM.