GLM-5.2 – How to Run Locally
Points and comments are a snapshot, not live.
Unsloth released dynamic GGUFs for GLM-5.2, allowing local inference of the 744B-parameter model.
GLM-5.2 (Z.ai) is a 744B parameter open model with 40B active and a 1M context window. Unsloth's dynamic quantization (1-bit to 8-bit) enables local running. 2-bit UD-IQ2_M uses ~239GB disk and fits on a 256GB unified memory Mac or 1x24GB GPU with 256GB RAM. Benchmarks show dynamic 1-bit achieves ~76% top-1 accuracy while being 86% smaller than the full 1.5TB model. The model supports three thinking modes (non-thinking, High, Max). Unsloth Studio and llama.cpp inference instructions are provided.
What commenters are saying
Hardware requirements dominate the thread. A commenter with 192GB RAM + RTX 3090 24GB notes they almost meet the usage guide's specs (24GB VRAM, 256GB RAM for MoE offloading). Another estimates generation speed as memory bandwidth-limited: at 4-bit quant with ~40B active parameters (~20GB), 100GB/s yields ~5 tokens/s, doublable with speculation. Cost estimates range from ~$50k-90k for decent throughput setups to $500k for unquantized. Users warn that KL-divergence lossless claims (e.g., Q4) may not hold for real long-context tasks, recommending going a couple steps higher.