LiteLSTM Architecture Based on Weights Sharing for Recurrent Neural Networks

TL;DR

This paper introduces LiteLSTM, a lightweight recurrent neural network architecture that reduces computational costs through weight sharing, maintaining performance for applications with limited hardware resources.

Contribution

The paper presents a novel LiteLSTM architecture that decreases computation by sharing weights, enabling efficient processing in resource-constrained environments without sacrificing accuracy.

Findings

LiteLSTM achieves comparable accuracy to state-of-the-art RNNs.

It significantly reduces computational costs and resource usage.

Effective across diverse domains like vision, cybersecurity, and speech recognition.

Abstract

Long short-term memory (LSTM) is one of the robust recurrent neural network architectures for learning sequential data. However, it requires considerable computational power to learn and implement both software and hardware aspects. This paper proposed a novel LiteLSTM architecture based on reducing the LSTM computation components via the weights sharing concept to reduce the overall architecture computation cost and maintain the architecture performance. The proposed LiteLSTM can be significant for processing large data where time-consuming is crucial while hardware resources are limited, such as the security of IoT devices and medical data processing. The proposed model was evaluated and tested empirically on three different datasets from the computer vision, cybersecurity, speech emotion recognition domains. The proposed LiteLSTM has comparable accuracy to the other state-of-the-art recurrent architecture while using a smaller computation budget.