Can gzip be a language model?
Points and comments are a snapshot, not live.
Using gzip's DEFLATE algorithm and beam search can generate text by finding the most compressible continuations.
The author builds `gzipt`, a pure-Python tool using zlib's DEFLATE compressor, to perform language modeling without neural networks. DEFLATE encodes byte sequences as cheap back-references when they match recent text in a 32 KiB sliding window. By scoring candidate continuations via compressed length and using beam search over byte sequences, the tool generates text that echoes the priming corpus. The paper "Language Modeling is Compression" establishes the compression-prediction equivalence underlying this approach. The author notes that only the last `tail` bytes of generated output stay in the scoring context to avoid verbatim loops.
What commenters are saying
Commenters largely view the experiment as a clever proof of concept but limited by DEFLATE's small window. A top comment demonstrates that naively copying text from the input context (using `rfind`) produces a better-compressing continuation than beam search, undermining the generative value. Another camp argues that gzip's linear scaling and lack of attention make it fundamentally different from LLMs, which handle diverse narratives. The Hutter Prize is cited as evidence that neural networks outperform traditional compressors, though model size disqualifies LLMs. A few commenters note the heavy tuning required to produce interesting output.