Training Neural Networks for Execution on Approximate Hardware

TL;DR

This paper explores specialized training methods for neural networks on approximate hardware, demonstrating significant speedups and addressing the gap in training approaches for energy-efficient deep learning inference.

Contribution

It introduces training techniques tailored for approximate hardware and proposes methods to accelerate training by up to 18 times.

Findings

Training methods need to be adapted for approximate hardware

Proposed techniques achieve up to 18X faster training

Enhances the practicality of approximate computing in deep learning

Abstract

Approximate computing methods have shown great potential for deep learning. Due to the reduced hardware costs, these methods are especially suitable for inference tasks on battery-operated devices that are constrained by their power budget. However, approximate computing hasn't reached its full potential due to the lack of work on training methods. In this work, we discuss training methods for approximate hardware. We demonstrate how training needs to be specialized for approximate hardware, and propose methods to speed up the training process by up to 18X.