R2N2: Residual Recurrent Neural Networks for Multivariate Time Series Forecasting

TL;DR

The paper introduces R2N2, a hybrid residual recurrent neural network model that combines linear VAR modeling with RNNs to improve multivariate time series forecasting accuracy and training efficiency.

Contribution

The paper proposes R2N2, a novel hybrid model that effectively combines VAR and RNN approaches for better multivariate time series prediction.

Findings

R2N2 outperforms VAR and RNN alone on real datasets.

R2N2 trains faster and uses fewer hidden units than RNNs.

R2N2 achieves competitive or superior accuracy in forecasting.

Abstract

Multivariate time-series modeling and forecasting is an important problem with numerous applications. Traditional approaches such as VAR (vector auto-regressive) models and more recent approaches such as RNNs (recurrent neural networks) are indispensable tools in modeling time-series data. In many multivariate time series modeling problems, there is usually a significant linear dependency component, for which VARs are suitable, and a nonlinear component, for which RNNs are suitable. Modeling such times series with only VAR or only RNNs can lead to poor predictive performance or complex models with large training times. In this work, we propose a hybrid model called R2N2 (Residual RNN), which first models the time series with a simple linear model (like VAR) and then models its residual errors using RNNs. R2N2s can be trained using existing algorithms for VARs and RNNs. Through an extensive empirical evaluation on two real world datasets (aviation and climate domains), we show that R2N2 is competitive, usually better than VAR or RNN, used alone. We also show that R2N2 is faster to train as compared to an RNN, while requiring less number of hidden units.