GLM-5.2 744B MoE runs on 25GB RAM PC via disk streaming
Original: Show HN: Getting GLM 5.2 running on my slow computer
Why This Matters
Demonstrates that frontier-scale MoE models can run on consumer hardware through selective disk streaming, lowering access barriers significantly.
Developer JustVugg released 'colibri,' an open-source engine written in pure C with zero dependencies that runs the 744B-parameter GLM-5.2 MoE model on a consumer machine with approximately 25GB of RAM by streaming expert weights from disk on demand.
JustVugg published 'colibri' on GitHub, a lightweight inference engine that enables running GLM-5.2 — a 744-billion-parameter Mixture-of-Experts (MoE) model — on consumer hardware with around 25GB of RAM. The project is implemented in pure C with no external dependencies.
The core insight is that a MoE model only activates a fraction of its parameters per token. GLM-5.2 activates roughly 40B parameters per token, of which only about 11GB changes between tokens (the routed experts). Colibri keeps the dense components — attention layers, shared experts, and embeddings (~17B params) — resident in RAM at int4 quantization, consuming approximately 9.9GB. The remaining 21,504 routed experts (75 MoE layers × 256 experts plus an MTP head, ~19MB each at int4) are stored on disk at roughly 370GB total and streamed on demand using a per-layer LRU cache. The engine reportedly becomes ready in 32 seconds. The repository has already accumulated over 837 stars on GitHub and 51 forks as of publication.