E-PUR: An Energy-Efficient Processing Unit for Recurrent Neural Networks

TL;DR

E-PUR is a specialized energy-efficient processing unit designed for LSTM-based RNNs, significantly reducing energy consumption and improving real-time performance on mobile devices through innovative memory locality techniques.

Contribution

The paper introduces E-PUR, a novel hardware architecture for LSTM RNNs, featuring Maximizing Weight Locality (MWL) to enhance memory efficiency and energy savings.

Findings

Achieves real-time LSTM performance on mobile devices.

Reduces energy consumption by up to 92x compared to NVIDIA Tegra X1.

Requires a very small chip area.

Abstract

Recurrent Neural Networks (RNNs) are a key technology for emerging applications such as automatic speech recognition, machine translation or image description. Long Short Term Memory (LSTM) networks are the most successful RNN implementation, as they can learn long term dependencies to achieve high accuracy. Unfortunately, the recurrent nature of LSTM networks significantly constrains the amount of parallelism and, hence, multicore CPUs and many-core GPUs exhibit poor efficiency for RNN inference. In this paper, we present E-PUR, an energy-efficient processing unit tailored to the requirements of LSTM computation. The main goal of E-PUR is to support large recurrent neural networks for low-power mobile devices. E-PUR provides an efficient hardware implementation of LSTM networks that is flexible to support diverse applications. One of its main novelties is a technique that we call Maximizing Weight Locality (MWL), which improves the temporal locality of the memory accesses for fetching the synaptic weights, reducing the memory requirements by a large extent. Our experimental results show that E-PUR achieves real-time performance for different LSTM networks, while reducing energy consumption by orders of magnitude with respect to general-purpose processors and GPUs, and it requires a very small chip area. Compared to a modern mobile SoC, an NVIDIA Tegra X1, E-PUR provides an average energy reduction of 92x.