Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint
Points and comments are a snapshot, not live.
Prism ML's Bonsai 2 27B model uses ternary weights for near-lossless compression to 5.9 GB.
Bonsai 2 27B, based on Qwen3.8 27B, uses ternary {−1, 0, +1} weights with FP16 scaling for 1.76 effective bits per weight, achieving a 5.9 GB footprint. It retains 98.2% of the full-precision model's aggregate benchmark performance while being 9x smaller. The model supports 262K-token context, multimodal input, and Apache 2.0 licensing. It runs at 143 tokens/second on an RTX 5090, 46.8 tokens/second on M5 Max, and consumes 40% less energy than a full-precision 8B model. Platform support includes NVIDIA CUDA and Apple MLX.
What commenters are saying
Commenters are excited about running a 27B-class model on hardware with limited VRAM, especially 16 GB GPUs. Many note that previous quants of Qwen3.8 27B already fit on 24 GB cards, so this release is most valuable for 16 GB systems. Several users share setup tips: the model requires Prism's llama.cpp fork, and it can be run via curl or Hugging Face download. Some report that longer tasks cause the model to "fall apart spectacularly and in interesting ways." There is interest in a future 8B version and contractor wider upstream llama.cpp support.