28.9M parameter LLM runs on $8 ESP32-S3 microcontroller
Original: Running a 28.9M parameter LLM on an $8 microcontroller
Why This Matters
Demonstrates a 100x parameter leap for on-device LLMs on ultra-low-cost embedded hardware, signaling new possibilities for offline edge AI.
Developer slvDev has demonstrated a 28.9 million parameter language model running locally on an ESP32-S3 microcontroller costing approximately $8, generating text at ~9 tokens per second on a small display with no server connection required.
GitHub user slvDev has published an open-source project called esp32-ai, which runs a 28.9 million parameter language model entirely on an ESP32-S3 microcontroller — a chip priced at around $8 with 512KB SRAM, 8MB PSRAM, and 16MB flash storage. The model generates text locally at approximately 9 tokens per second, outputting directly to a small wired screen with no data sent to any external server. The key technical breakthrough enabling this is Per-Layer Embeddings, a memory optimization technique derived from Google's Gemma model architecture. Of the 28.9 million parameters, 25 million reside in a flash-based lookup table rather than in RAM, allowing the model to fit within the chip's constrained memory footprint. The project claims this is roughly 100 times larger than the previous record for a language model running on comparable hardware, which stood at approximately 260,000 parameters. The repository includes firmware, experiments, source code, and result benchmarks, and has attracted over 900 GitHub stars since publication.