Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networks

TL;DR

This paper provides a comprehensive tutorial on LSTM-RNNs, explaining their evolution, functioning, and improvements in documentation, aiming to clarify their impressive capabilities and correct previous inconsistencies.

Contribution

It offers an improved, unified explanation of LSTM-RNNs, correcting errors and enhancing understanding of their development and operation.

Findings

Clarified the evolution of LSTM-RNNs

Fixed errors in previous publications

Unified notation for better understanding

Abstract

Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN) are one of the most powerful dynamic classifiers publicly known. The network itself and the related learning algorithms are reasonably well documented to get an idea how it works. This paper will shed more light into understanding how LSTM-RNNs evolved and why they work impressively well, focusing on the early, ground-breaking publications. We significantly improved documentation and fixed a number of errors and inconsistencies that accumulated in previous publications. To support understanding we as well revised and unified the notation used.