CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs

TL;DR

CMSIS-NN provides optimized neural network kernels for Arm Cortex-M CPUs, significantly enhancing performance and energy efficiency for IoT edge devices running neural network inference.

Contribution

The paper introduces CMSIS-NN, a set of optimized kernels that improve neural network inference performance and reduce memory usage on Cortex-M processors.

Findings

4.6X improvement in runtime/throughput

4.9X improvement in energy efficiency

Enhanced suitability for IoT edge devices

Abstract

Deep Neural Networks are becoming increasingly popular in always-on IoT edge devices performing data analytics right at the source, reducing latency as well as energy consumption for data communication. This paper presents CMSIS-NN, efficient kernels developed to maximize the performance and minimize the memory footprint of neural network (NN) applications on Arm Cortex-M processors targeted for intelligent IoT edge devices. Neural network inference based on CMSIS-NN kernels achieves 4.6X improvement in runtime/throughput and 4.9X improvement in energy efficiency.