Retrospectively Reverse-Engineering Apple's Neural Engine
Points and comments are a snapshot, not live.
Apple's Neural Engine was designed for CNNs, not transformers, limiting its utility.
The author reverse-engineers the M1 ANE, finding 16 compute cores with 2048 MAC lanes optimized for dense CNN dataflow with predictable reuse. Transformers broke this assumption, leading Apple to fold ANE cores into GPUs starting with M5 (2025). The scheduler uses fixed-size task descriptors, and the hardware lacks dynamic virtual addressing. The post-MAC activation block uses a 33-entry piecewise-linear LUT for tanh, with folding of bias/scale into convolution weights.
What commenters are saying
Commenters largely agree the ANE was designed for CNNs, not transformers, with several noting its limited real-world use despite Apple touting it. Some defend Apple's early investment, pointing to Face ID, crash detection, and Photos features running on the ANE. A key correction: the M5+ GPU's Neural Accelerators (NAX) are distinct from the ANE, which continues in M6/A20. One commenter ported a transformer to ANE by pretending it was a CNN. Another notes Apple's new Core AI framework may expand ANE usage.