Saving another 100TB of RAM
Points and comments are a snapshot, not live.
Cloudflare reclaimed 100TB+ of RAM by optimizing consistent hashing and its Rust structs.
Cloudflare's Pingora Backend Router used excessive memory due to many consistent hash rings for server subsets. Engineers reduced memory 25% by shrinking a `Point` struct from 8 bytes to 6 bytes using a raw byte array in Rust. A deeper analysis of the coefficient of variation showed diminishing returns from adding more hashes per server, with collision errors from 32-bit hashes. This allowed culling unnecessary hashes without degrading load balance, saving over 100TB globally on top of a prior DNS optimization.
What commenters are saying
Several readers praised the article's writing and math depth. A debate arose on whether large companies become impenetrable silos; some argued each abstraction layer exists for valid security and complexity reasons, while others lamented the overhead. One commenter noted Cloudflare's focus on performance is natural since hardware is their primary cost. Another pointed out the structural optimization is clever but questioned if the problem itself stems from excessive hashes initially. The thread also touched on AI's role in code exploration and the nostalgia of constrained optimization.