Apple Neural Engine: A Retrospective Reverse-Engineering
Original: Retrospectively Reverse-Engineering Apple's Neural Engine
Why This Matters
Documents how CNN-era NPU design assumptions collapsed under transformer workloads, from someone who built the driver.
Researcher Eileen Yoon revisited her abandoned Apple Neural Engine (ANE) reverse-engineering project three years later, mapping the M1 ANE's full internal architecture—compute, datapath, scheduler, memory, and execution model—to understand what Apple's 2017 CNN-era silicon assumptions reveal about the shift to transformer workloads.
Eileen Yoon originally stopped working on an open-source Linux ANE driver after concluding the hardware was too narrowly designed for general-purpose acceleration—Apple itself only uses the ANE regularly to generate upsampled Finder preview images in macOS. The M5 chip (2025) made the decision feel final: Apple folded ANE cores inside the GPU, signaling the standalone NPU's decline as LLM workloads dominate.
With the ANE effectively deprecated, Yoon returned not to make it useful but to document its architecture fully. The M1 ANE has 16 parallel compute cores, each with 128 FP16 (or 256 INT8) multiply-accumulate (MAC) lanes—2,048 MAC lanes total. Each lane runs a standard multiply-accumulate recurrence, making it structurally suited to dot products regardless of whether the workload is CNN convolution or transformer attention.
The real specialization, she argues, was never the MACs themselves but the dataflow surrounding them. Apple engineered predictable, reuse-optimized data movement for dense CNN inference—the kind of workload that was mainstream when the A11 Bionic introduced the ANE in 2017. Autoregressive transformer decode broke that assumption: access patterns became irregular, reuse became sparse, and the ANE's fixed dataflow couldn't adapt. The M5's architectural choice—absorbing NPU compute into GPU fabric—reflects exactly that mismatch playing out over eight years of silicon.