Unified Implementations of Recurrent Neural Networks in Multiple Deep Learning Frameworks

TL;DR

This paper introduces three open-source libraries in Julia and Python that unify the implementation of various RNN architectures, facilitating easier testing, customization, and reproducibility in sequence modeling tasks.

Contribution

The paper presents a unified framework with open-source libraries that centralize diverse RNN implementations, improving accessibility and reproducibility.

Findings

Libraries support multiple RNN variants

Framework enables easy customization and extension

Open-source and actively maintained

Abstract

Recurrent neural networks (RNNs) are a cornerstone of sequence modeling across various scientific and industrial applications. Owing to their versatility, numerous RNN variants have been proposed over the past decade, aiming to improve the modeling of long-term dependencies and to address challenges such as vanishing and exploding gradients. However, no central library is available to test these variations, and reimplementing diverse architectures can be time-consuming and error-prone, limiting reproducibility and exploration. Here, we introduce three open-source libraries in Julia and Python that centralize numerous recurrent cell implementations and higher-level recurrent architectures. torchrecurrent, RecurrentLayers.jl, and LuxRecurrentLayers.jl offer a consistent framework for constructing and extending RNN models, providing built-in mechanisms for customization and experimentation. All packages are available under the MIT license and actively maintained on GitHub.