LiteLSTM Architecture for Deep Recurrent Neural Networks

TL;DR

This paper introduces LiteLSTM, a computationally efficient variant of traditional LSTM that reduces complexity through weight sharing, suitable for big data applications like IoT security and medical data analysis.

Contribution

It presents a novel LiteLSTM architecture that maintains performance while significantly reducing computational costs using weight sharing techniques.

Findings

Reduces computation and energy consumption compared to standard LSTM

Maintains comparable performance on vision and cybersecurity datasets

Potentially lowers CO2 footprint of deep learning models

Abstract

Long short-term memory (LSTM) is a robust recurrent neural network architecture for learning spatiotemporal sequential data. However, it requires significant computational power for learning and implementing from both software and hardware aspects. This paper proposes a novel LiteLSTM architecture based on reducing the computation components of the LSTM using the weights sharing concept to reduce the overall architecture cost and maintain the architecture performance. The proposed LiteLSTM can be significant for learning big data where time-consumption is crucial such as the security of IoT devices and medical data. Moreover, it helps to reduce the CO2 footprint. The proposed model was evaluated and tested empirically on two different datasets from computer vision and cybersecurity domains.