Needle 2: 14MB agentic LLM for phones and wearables
Original: Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
Why This Matters
On-device agentic AI at 14MB could unlock AI functionality across billions of low-cost IoT and edge devices.
Cactus Compute released Needle 2, a 45M-parameter agentic LLM weighing just 14MB that runs on devices like Raspberry Pi 5, Samsung A-Series phones, and ESP32-S3 microcontrollers. It achieves 500+ tokens/sec decode on Pi 5 and uses only 28MB of RAM per session.
Cactus Compute has launched Needle 2, an open-source 45M-parameter language model designed specifically for tool calling, device control, and structured data extraction on resource-constrained hardware. The entire model is a single 14MB binary, compressed using the company's proprietary CQ2-bit Cactus Quants technique and built on their Simple Attention Network architecture. It runs a full session within 28MB of RAM, making it compatible with microcontrollers such as the ESP32-S3.
On benchmarks for tool calling and mobile device use, Needle 2 competes with models 5× to 70× larger, including FunctionGemma 270M, LFM2.5 230M, and Apple FM, while operating at only 2-bit precision versus their float16. Decode speed reaches 500 tokens/sec on Raspberry Pi 5 and 300–700 tokens/sec on sub-$200 Android phones like the Samsung A-Series. VR headsets such as Meta Quest 3S and Apple Vision Pro reach 400–1,500 tokens/sec.
The model targets the estimated 21 billion connected IoT devices worldwide, with approximately four in five edge devices costing under $200. Needle 2 is licensed under Apache 2.0, with weights available on Hugging Face. A live browser-based sandbox runs the model via WebAssembly.