Combining Gradients and Probabilities for Heterogeneous Approximation of Neural Networks

TL;DR

This paper introduces a method combining gradient-based training and probabilistic error modeling to efficiently identify energy-saving approximate multipliers for neural networks, maintaining high accuracy.

Contribution

It presents a novel approach that uses additive Gaussian noise as a surrogate for hardware approximation errors, enabling training-time optimization of energy-efficient multipliers.

Findings

Achieves 70-79% energy reduction in ResNet models on CIFAR-10 with minimal accuracy loss.

Reduces energy consumption by 53% in VGG16 on Tiny ImageNet with negligible accuracy drop.

Accurately predicts approximate multiplier parameters using the proposed error model.

Abstract

This work explores the search for heterogeneous approximate multiplier configurations for neural networks that produce high accuracy and low energy consumption. We discuss the validity of additive Gaussian noise added to accurate neural network computations as a surrogate model for behavioral simulation of approximate multipliers. The continuous and differentiable properties of the solution space spanned by the additive Gaussian noise model are used as a heuristic that generates meaningful estimates of layer robustness without the need for combinatorial optimization techniques. Instead, the amount of noise injected into the accurate computations is learned during network training using backpropagation. A probabilistic model of the multiplier error is presented to bridge the gap between the domains; the model estimates the standard deviation of the approximate multiplier error, connecting solutions in the additive Gaussian noise space to actual hardware instances. Our experiments show that the combination of heterogeneous approximation and neural network retraining reduces the energy consumption for multiplications by 70% to 79% for different ResNet variants on the CIFAR-10 dataset with a Top-1 accuracy loss below one percentage point. For the more complex Tiny ImageNet task, our VGG16 model achieves a 53 % reduction in energy consumption with a drop in Top-5 accuracy of 0.5 percentage points. We further demonstrate that our error model can predict the parameters of an approximate multiplier in the context of the commonly used additive Gaussian noise (AGN) model with high accuracy. Our software implementation is available under https://github.com/etrommer/agn-approx.