ARMA Cell: A Modular and Effective Approach for Neural Autoregressive Modeling

TL;DR

The paper introduces ARMA and ConvARMA cells as simple, modular neural network components for time series modeling, offering competitive performance with increased robustness and versatility over complex RNNs.

Contribution

It proposes the ARMA cell and its convolutional variant as effective, modular alternatives to traditional RNNs for neural time series modeling.

Findings

ARMA cell performs competitively with RNNs.

ARMA cell is more robust and simpler to implement.

ConvARMA extends to spatially-correlated data.

Abstract

The autoregressive moving average (ARMA) model is a classical, and arguably one of the most studied approaches to model time series data. It has compelling theoretical properties and is widely used among practitioners. More recent deep learning approaches popularize recurrent neural networks (RNNs) and, in particular, Long Short-Term Memory (LSTM) cells that have become one of the best performing and most common building blocks in neural time series modeling. While advantageous for time series data or sequences with long-term effects, complex RNN cells are not always a must and can sometimes even be inferior to simpler recurrent approaches. In this work, we introduce the ARMA cell, a simpler, modular, and effective approach for time series modeling in neural networks. This cell can be used in any neural network architecture where recurrent structures are present and naturally handles multivariate time series using vector autoregression. We also introduce the ConvARMA cell as a natural successor for spatially-correlated time series. Our experiments show that the proposed methodology is competitive with popular alternatives in terms of performance while being more robust and compelling due to its simplicity