Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
Points and comments are a snapshot, not live.
TurboFieldfare runs Gemma 4 26B-A4B in ~2 GB RAM on any M-series Mac.
A custom Swift + Metal runtime streams only the needed experts from SSD, keeping a 1.35 GB core and KV cache in memory. On an 8 GB M2 MacBook Air it achieves 5.1-6.3 tok/s; on a 24 GB M5 Pro, 31-35 tok/s. The installer repacks the 14.3 GB model directly into a .gturbo layout without staging the full checkpoint. Includes a native Mac app, CLI, and loopback OpenAI-compatible server. The project documents 103 measured experiments across kernels, caching, and I/O.
The engine uses an LFU expert cache with a ~67% hit rate, overlapping SSD reads with GPU computation. The author notes the model is text-only and not affiliated with Google.
What commenters are saying
Commenters were impressed by the memory efficiency and expert caching strategy. The author answered questions in detail: the full router changes almost every token, but ~40% of experts repeat on the next token and ~57% within two tokens, cutting I/O from 166 to 88 ms/token on M2. They compared mmap vs pread: mmap gave 0.50 tok/s while pread achieved 4 tok/s on the same hardware.
Several commenters asked about comparisons with llama.cpp and DwarfStar4. The author noted llama.cpp already runs 26B in 2GB via mmap but likely slower. A commenter confirmed it works on an M1 MacBook Air (4-5 tok/s) and on a MacBook Neo (4.5 tok/s). The author clarified it is Apple-only due to Metal and shared memory architecture.