Long Short-Term Memory Implementation Exploiting Passive RRAM Crossbar Array

TL;DR

This paper presents a novel, highly efficient hardware implementation of LSTM networks using passive RRAM crossbar arrays, significantly reducing area and energy consumption while maintaining robustness and accuracy.

Contribution

It introduces the first passive RRAM crossbar-based LSTM implementation, demonstrating superior efficiency and robustness compared to prior digital and active crossbar approaches.

Findings

Outperforms prior implementations by over three orders of magnitude in area.

Reduces training energy by two orders of magnitude.

Shows robustness against hardware variations and noise.

Abstract

The ever-increasing demand to extract temporal correlations across sequential data and perform context-based learning in this era of big data has led to the development of long short-term memory (LSTM) networks. Furthermore, there is an urgent need to perform these time-series data-dependent applications including speech/video processing and recognition, language modelling and translation, etc. on compact internet-of-things (IoT) edge devices with limited energy. To this end, in this work, for the first time, we propose an extremely area- and energy-efficient LSTM network implementation exploiting the passive resistive random access memory (RRAM) crossbar array. We developed a hardware-aware LSTM network simulation framework and performed an extensive analysis of the proposed LSTM implementation considering the non-ideal hardware artifacts such as spatial (device-to-device) and temporal variations, non-linearity, noise, etc. utilizing an experimentally calibrated comprehensive phenomenological model for passive RRAM crossbar array. Our results indicate that the proposed passive RRAM crossbar-based LSTM network implementation not only outperforms the prior digital and active 1T-1R crossbar-based LSTM implementations by more than three orders of magnitude in terms of area and two orders of magnitude in terms of training energy for identical network accuracy, but also exhibits robustness against spatial and temporal variations and noise, and a faster convergence rate. Our work may provide the incentive for experimental realization of LSTM networks on passive RRAM crossbar arrays.