Gemma 4 26B runs at 5 tokens/sec on a 13-year-old CPU with no GPU
Original: Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU
Why This Matters
Demonstrates that capable open-weight LLMs can run locally on decade-old enterprise hardware, lowering the barrier to private AI deployment.
Engineer Ryan Findley ran Google's Gemma 4 26B mixture-of-experts model at ~5.2 tokens/sec on a 2013-era dual Xeon E5-2690 v2 server with no GPU, using ik_llama.cpp and a custom AVX1-only patch developed with Claude AI assistance.
Ryan Findley of Neomind Labs successfully ran Google's Gemma 4 26B-A4B (MoE, Q8_0) on a repurposed HP StoreVirtual storage server equipped with dual Intel Ivy Bridge Xeon E5-2690 v2 CPUs (2013), DDR3 RAM, and no GPU — a machine purchased for under $300. The server achieved approximately 5.2 tokens/sec decode speed and ~16 tokens/sec prompt evaluation, described as 'reading speed.' The project was inspired by a Hacker News post about running Gemma 4 on a 2016-era Broadwell Xeon. Findley's older Ivy Bridge chips lack AVX2 and FMA3 instruction sets, which the ik_llama.cpp fork's optimized kernels depend on. When the build failed, Findley used Anthropic's Claude to diagnose the microarchitecture incompatibility and rework the hot paths to fall back gracefully on pre-AVX2 hardware without discarding other performance optimizations. The resulting patch has been submitted as pull request ikawrakow/ik_llama.cpp#2138 and is awaiting maintainer review. Findley emphasized that the key skill involved was understanding model and hardware deeply enough to bridge the gap — not simply paying for cloud GPU access.