Approximate Multiplier Induced Error Propagation in Deep Neural Networks

TL;DR

This paper develops an analytical framework linking approximate multiplier errors to DNN accuracy degradation, enabling rapid estimation of error impacts without extensive simulations.

Contribution

It introduces a mathematical model that predicts DNN distortion from AxM error moments, validated through FPGA implementation and error injection experiments.

Findings

Distortion mainly governed by multiplier bias error

Model accurately predicts accuracy degradation on ImageNet networks

FPGA case study confirms analytical predictions

Abstract

Deep Neural Networks (DNNs) rely heavily on dense arithmetic operations, motivating the use of Approximate Multipliers (AxMs) to reduce energy consumption in hardware accelerators. However, a rigorous mathematical characterization of how AxMs error distributions influence DNN accuracy remains underdeveloped. This work presents an analytical framework that connects the statistical error moments of an AxM to the induced distortion in General Matrix Multiplication (GEMM). Using the Frobenius norm of the resulting error matrix, we derive a closed form expression for practical DNN dimensions that demonstrates the distortion is predominantly governed by the multiplier mean error (bias). To evaluate this model in realistic settings, we incorporate controlled error injection into GEMM and convolution layers and examine its effect on ImageNet scale networks. The predicted distortion correlates strongly with the observed accuracy degradation, and an error configurable AxM case study implemented on an FPGA further confirms the analytical trends. By providing a lightweight alternative to behavioral or hardware level simulations, this framework enables rapid estimation of AxM impact on DNN inference quality.