Approximations in Deep Learning

TL;DR

This paper explores how Approximate Computing can enhance the performance and energy efficiency of deep learning models, especially on resource-constrained devices, by relaxing precision requirements during inference and training.

Contribution

It provides an analysis of how AxC techniques can be applied to deep learning to improve efficiency without significantly compromising accuracy.

Findings

AxC improves energy efficiency in DL hardware accelerators.

Relaxing precision can maintain acceptable accuracy levels.

AxC techniques benefit both inference and training processes.

Abstract

The design and implementation of Deep Learning (DL) models is currently receiving a lot of attention from both industrials and academics. However, the computational workload associated with DL is often out of reach for low-power embedded devices and is still costly when run on datacenters. By relaxing the need for fully precise operations, Approximate Computing (AxC) substantially improves performance and energy efficiency. DL is extremely relevant in this context, since playing with the accuracy needed to do adequate computations will significantly enhance performance, while keeping the quality of results in a user-constrained range. This chapter will explore how AxC can improve the performance and energy efficiency of hardware accelerators in DL applications during inference and training.