QL-LSTM: A Parameter-Efficient LSTM for Stable Long-Sequence Modeling

TL;DR

The paper introduces QL-LSTM, a more parameter-efficient recurrent neural network that maintains performance on long sequences by reducing parameters and enhancing information flow, with potential for faster processing after further optimization.

Contribution

It proposes two novel components, Parameter-Shared Unified Gating and Hierarchical Gated Recurrence with Additive Skip Connections, to improve efficiency and long-range information retention in LSTM models.

Findings

QL-LSTM reduces parameters by ~48% while maintaining accuracy.

It improves long-range information flow with additive skip connections.

Achieves competitive sentiment classification results on IMDB dataset.

Abstract

Recurrent neural architectures such as LSTM and GRU remain widely used in sequence modeling, but they continue to face two core limitations: redundant gate-specific parameters and reduced ability to retain information across long temporal distances. This paper introduces the Quantum-Leap LSTM (QL-LSTM), a recurrent architecture designed to address both challenges through two independent components. The Parameter-Shared Unified Gating mechanism replaces all gate-specific transformations with a single shared weight matrix, reducing parameters by approximately 48 percent while preserving full gating behavior. The Hierarchical Gated Recurrence with Additive Skip Connections component adds a multiplication-free pathway that improves long-range information flow and reduces forget-gate degradation. We evaluate QL-LSTM on sentiment classification using the IMDB dataset with extended document lengths, comparing it to LSTM, GRU, and BiLSTM reference models. QL-LSTM achieves competitive accuracy while using substantially fewer parameters. Although the PSUG and HGR-ASC components are more efficient per time step, the current prototype remains limited by the inherent sequential nature of recurrent models and therefore does not yet yield wall-clock speed improvements without further kernel-level optimization.