Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone
Points and comments are a snapshot, not live.
Swiftlet runs 35B and 80B Qwen models on Mac and iPhone by streaming MoE weights from storage.
Swiftlet is a Swift + Metal runtime for Qwen3-Next and Qwen3.5/3.6 MoE hybrid models. It keeps only the dense core in RAM and streams routed experts from disk via pread, enabling an 80B Qwen in 4.3 GB of RAM on an M5 Mac at 4.5-5 tok/s. The 35B model uses 2.6 GB RAM and runs on an iPhone 17 at ~1 tok/s. Only about 3B parameters activate per token, so models chat like large models but recall facts like small ones. Includes CLI, OpenAI-compatible server, and open-source iOS app. Apache 2.0 license.
What commenters are saying
Commenters split into skeptics and optimists. Skeptics noted that prefill becomes the bottleneck, with half an hour to process 10k tokens on an M5, and that renting a GPU is more economical. Others countered that this is how progress happens, with foundational work enabling future speedups. Several commenters dismissed SSD wear concerns, noting only writes degrade NAND and read disturb is rare in practice. One commenter reported getting 86 tok/s on a 12B Q6_K model entirely in VRAM on a 4090, highlighting the gap between local and dedicated hardware.