Simplified Long Short-term Memory Recurrent Neural Networks: part III

TL;DR

This paper introduces two new simplified LSTM variants that significantly reduce computational complexity while maintaining performance, addressing the need for efficient processing of large time-sequence data in resource-constrained environments.

Contribution

It presents and evaluates two novel simplified LSTM models with pointwise state multiplications, enhancing computational efficiency without sacrificing accuracy.

Findings

New LSTM variants reduce computational load

Performance comparable to standard LSTM models

Effective for processing large time-sequence data

Abstract

This is part III of three-part work. In parts I and II, we have presented eight variants for simplified Long Short Term Memory (LSTM) recurrent neural networks (RNNs). It is noted that fast computation, specially in constrained computing resources, are an important factor in processing big time-sequence data. In this part III paper, we present and evaluate two new LSTM model variants which dramatically reduce the computational load while retaining comparable performance to the base (standard) LSTM RNNs. In these new variants, we impose (Hadamard) pointwise state multiplications in the cell-memory network in addition to the gating signal networks.