Evaluation of Domain-Specific Architectures for General-Purpose Applications in Apple Silicon

TL;DR

This paper assesses Apple's Neural Engine for general-purpose high-performance computing, demonstrating competitive performance and superior energy efficiency compared to GPUs when algorithms are properly adapted.

Contribution

It provides a comprehensive evaluation of ANE's potential for HPC workloads, highlighting its energy efficiency and performance capabilities in comparison to traditional GPUs.

Findings

ANE achieves up to 3.8 TFlops on M4-Pro

ANE consumes significantly less energy than GPU

Proper algorithm adaptation is crucial for performance

Abstract

The rise of AI and its growing computational demands have driven the integration of domain-specific accelerators (such as GPUs, TPUs, and NPUs) across the entire computing infrastructure. Following the precedent set by the GPGPU which popularized GPUs for general-purpose tasks, this research asks whether this phenomenon can be replicated with specialized accelerators like NPUs in new contexts. This paper evaluates the potential of the Apple Neural Engine (ANE) designed for high energy efficiency in Machine Learning workloads, in the context of general-purpose HPC applications. We evaluate the performance and energy consumption of classic HPC algorithms such as GEMM, Jacobi or Multigrid methods on Apple's ANE across the M1 and the latest M4 architectures. Results confirm that, when algorithms are properly adapted, the ANE achieves competitive performance (up to 3.8 TFlops on the M4-Pro, comparable to the GPU's 4.7 TFlops on the same SoC for GEMM operation) while demonstrating significantly superior energy efficiency (e.g., GEMM consumes 5.2 Watts on the ANE versus 24 Watts on GPU counterpart in M4 architectures).