A Construction Kit for Efficient Low Power Neural Network Accelerator Designs

TL;DR

This paper offers a comprehensive construction kit of optimization techniques for designing low-power neural network accelerators, enabling systematic evaluation of individual design choices to improve edge processing efficiency.

Contribution

It introduces a quantitative framework that isolates and assesses the effects of various optimization techniques in neural network accelerator designs.

Findings

Memory savings up to 10,000x

Energy reductions up to 33x

Framework for comparing optimization impacts

Abstract

Implementing embedded neural network processing at the edge requires efficient hardware acceleration that couples high computational performance with low power consumption. Driven by the rapid evolution of network architectures and their algorithmic features, accelerator designs are constantly updated and improved. To evaluate and compare hardware design choices, designers can refer to a myriad of accelerator implementations in the literature. Surveys provide an overview of these works but are often limited to system-level and benchmark-specific performance metrics, making it difficult to quantitatively compare the individual effect of each utilized optimization technique. This complicates the evaluation of optimizations for new accelerator designs, slowing-down the research progress. This work provides a survey of neural network accelerator optimization approaches that have been used in recent works and reports their individual effects on edge processing performance. It presents the list of optimizations and their quantitative effects as a construction kit, allowing to assess the design choices for each building block separately. Reported optimizations range from up to 10'000x memory savings to 33x energy reductions, providing chip designers an overview of design choices for implementing efficient low power neural network accelerators.