Hardware Approximate Techniques for Deep Neural Network Accelerators: A Survey

TL;DR

This survey reviews hardware approximation techniques for DNN accelerators, analyzing their types, evaluation complexity, and potential to improve energy efficiency, reliability, and security in neural network inference.

Contribution

It provides a comprehensive classification and analysis of hardware approximation methods for DNNs, including evaluation metrics and future research directions.

Findings

Identified key approximation families and their characteristics.

Assessed evaluation complexity and efficiency of approximate DNN accelerators.

Discussed error metrics, accuracy recovery, and broader benefits like security.

Abstract

Deep Neural Networks (DNNs) are very popular because of their high performance in various cognitive tasks in Machine Learning (ML). Recent advancements in DNNs have brought beyond human accuracy in many tasks, but at the cost of high computational complexity. To enable efficient execution of DNN inference, more and more research works, therefore, exploit the inherent error resilience of DNNs and employ Approximate Computing (AC) principles to address the elevated energy demands of DNN accelerators. This article provides a comprehensive survey and analysis of hardware approximation techniques for DNN accelerators. First, we analyze the state of the art and by identifying approximation families, we cluster the respective works with respect to the approximation type. Next, we analyze the complexity of the performed evaluations (with respect to the dataset and DNN size) to assess the efficiency, the potential, and limitations of approximate DNN accelerators. Moreover, a broad discussion is provided, regarding error metrics that are more suitable for designing approximate units for DNN accelerators as well as accuracy recovery approaches that are tailored to DNN inference. Finally, we present how Approximate Computing for DNN accelerators can go beyond energy efficiency and address reliability and security issues, as well.