uv: Deduplicate all files in the wheel cache

222 points · 113 comments on HN · read original →

Points and comments are a snapshot, not live.

uv deduplicates identical files across cached wheels by storing each file under its BLAKE3 hash.

When content-addressed caching is enabled in uv, overhead from hashing during wheel extraction is reduced by reusing a single 64 KiB buffer across the entire wheel instead of allocating one per file. This cuts buffer allocations from 11,120 to one for PyTorch wheels. Cold-install benchmarks show 7-10% speedups (e.g., PyTorch from 6.50s to 5.99s, NumPy from 627ms to 567ms). A follow-up on macOS speeds cache cleanup by using `getattrlistbulk` to batch-read hard-link counts, reducing median scan times from ~386ms to ~101-105ms.

What commenters are saying

Many commenters find uv's speed less important than its convenience as a single binary that manages Python versions, creates local .venvs, and supports pyproject.toml. Others argue pip matches uv for git-installs. A uv team member offers to investigate slow runs with verbose logs and warns that script lockfiles aren't automatic yet. A thread discusses BLAKE3 for file deduplication: it's a fast cryptographic hash that can serve as a file-identity signal, though one reader misread 40 hex digits as a 40-bit hash and was corrected about collision risk.