On the Impact of Device-Level Techniques on Energy-Efficiency of Neural Network Accelerators

TL;DR

This paper investigates device-level techniques like voltage, frequency, and data quantization to enhance energy-efficiency in FPGA and GPU accelerators for neural networks, aiming to optimize power use in edge computing.

Contribution

It experimentally compares device-level energy-efficiency techniques on FPGAs and GPUs, highlighting their impact on power, energy, and performance with minimal accuracy loss.

Findings

Frequency scaling reduces power and energy but affects performance.

Reduced-precision instructions improve power, energy, and performance with negligible accuracy loss.

GPU and FPGA benefits vary depending on the technique applied.

Abstract

Energy-efficiency is a key concern for neural network applications. To alleviate this issue, hardware acceleration using FPGAs or GPUs can provide better energy-efficiency than general-purpose processors. However, further improvement of the energy-efficiency of such accelerators will be extremely beneficial specially to deploy neural network in power-constrained edge computing environments. In this paper, we experimentally explore the potential of device-level energy-efficiency techniques (e.g.,supply voltage underscaling, frequency scaling, and data quantization) for representative off-the-shelf FPGAs compared to GPUs. Frequency scaling in both platforms can improve the power and energy consumption but with performance overhead, e.g.,in GPUs it improves the power consumption and GOPs/J by up to 34% and 28%, respectively. However, leveraging reduced-precision instructions improves power (up to 13%), energy (up to 20%), and performance (up to 7%) simultaneously, with negligible reduction in accuracy of neural network accuracy.