ParaRNN: An Interpretable and Parallelizable Recurrent Neural Network for Time-Dependent Data

TL;DR

ParaRNN introduces a parallelizable, interpretable RNN variant that maintains strong predictive performance while enhancing understanding and computational efficiency in time-dependent data modeling.

Contribution

It proposes ParaRNN, a novel RNN architecture with additive, interpretable components and parallelizable design, bridging the gap between interpretability and performance.

Findings

ParaRNN achieves comparable accuracy to standard RNNs.

It offers improved interpretability through additive component representation.

Empirical results demonstrate efficiency gains in sequential tasks.

Abstract

The proliferation of large-scale and structurally complex data has spurred the integration of machine learning methods into statistical modeling. Recurrent neural networks (RNNs), a foundational class of models for time-dependent data, can be viewed as nonlinear extensions of classical autoregressive moving average models. Despite their flexibility and empirical success in machine learning, RNNs often suffer from limited interpretability and slow training, which hinders their use in statistics. This paper proposes the Parallelized RNN (ParaRNN), a novel model composed of multiple small recurrent units. ParaRNN admits an additive representation that decouples recurrent dynamics into interpretable components, whose behavior can be characterized through recurrence features. This interpretability enables its applications in nonparametric regression for time-dependent data, while the design also allows efficient parallelization. The approximation capacity and non-asymptotic prediction error bounds in a nonparametric regression setting are established for ParaRNN. Empirical results on three sequential modeling tasks further demonstrate that ParaRNN achieves performance comparable to vanilla RNNs while offering improved interpretability and efficiency.