My local model setup on an M4 Pro Mac Mini
Points and comments are a snapshot, not live.
An M4 Pro Mac Mini runs local LLMs, using MoE models to fit 35B parameters in 48GB RAM.
The author runs a local LLM server on an M4 Pro Mac Mini (48GB RAM) using oMLX and Tailscale. They use Qwen3.6-35B-A3B-OptiQ-4bit (MoE, ~20GB RAM) for reasoning tasks and Gemma-4-E4B-it-OptiQ-4bit (~2.4GB) for simple chats. The MoE model activates 3B of 35B total parameters per token, making it practical on 48GB. Performance: ~325 tok/s prompt processing, ~34 tok/s generation. The author cites cost predictability, data privacy, offline capability, and AI sovereignty as reasons for local inference. They access models via Hermes agent, Apollo iOS, Raycast, and Pi, all connected through Tailscale.
What commenters are saying
Commenters were split on local LLM practicality. One camp argued that local models offer no performance advantage over API-based frontier models, and that hardware costs rarely break even. Another camp defended local use for specific tasks: offline information retrieval, Rust learning, data reformatting, translation, and summarization-all at indefinite free usage. A commenter with an M4 Pro 48GB reported 52 tok/s on Gemma 4 26b and 72 tok/s on Qwen 3.5b (4-bit). One user noted that local coding agents required significantly more API tokens for fixes, negating savings. Another detailed the economics: a $200k+ cluster can be cheaper than Anthropic's scale tier for heavy 24/7 usage.