I'm making LLMs work on small and old devices
Points and comments are a snapshot, not live.
Article body wasn't reachable. The HN discussion summary is below.
Points and comments are a snapshot, not live.
Article body wasn't reachable. The HN discussion summary is below.
What commenters are saying
The thread is enthusiastic about running LLMs on obsolete or constrained hardware. The author says they have reached 16B models running on an iPad Air 1st gen and a Xiaomi TV box (both under 1GB usable memory), and open-sourced kernels and modified models on Hugging Face. Commenters suggest challenging targets: Playdate, vapes with screens, PlayStation 2 dev kit, Raspberry Pi Zero, ESP32, and old Pentium machines. Some debate the utility of small models vs. the importance of making AI less resource-hungry. Several links to prior work: PS2 LLM, Nintendo Wii LLM, ESP32 per-layer embedding streaming. A few commenters discuss actual applications like pihole, weather station, Terraria servers on old thin clients.