Fast and Interpretable Autoregressive Estimation with Neural Network Backpropagation

TL;DR

This paper introduces a neural network-based method for autoregressive model estimation that is faster, more reliable when traditional methods fail, and maintains interpretability, demonstrated through extensive simulations.

Contribution

It embeds AR structure into a neural network, enabling efficient coefficient estimation via backpropagation while preserving interpretability, outperforming traditional methods in convergence and speed.

Findings

NN method reliably recovers AR coefficients in all simulations.

Compared to CML, NN achieves median 12.6x speedup, up to 34.2x.

NN maintains comparable accuracy when CML converges, and succeeds when CML fails.

Abstract

Autoregressive (AR) models remain widely used in time series analysis due to their interpretability, but convencional parameter estimation methods can be computationally expensive and prone to convergence issues. This paper proposes a Neural Network (NN) formulation of AR estimation by embedding the autoregressive structure directly into a feedforward NN, enabling coefficient estimation through backpropagation while preserving interpretability. Simulation experiments on 125,000 synthetic AR(p) time series with short-term dependence (1 <= p <= 5) show that the proposed NN-based method consistently recovers model coefficients for all series, while Conditional Maximum Likelihood (CML) fails to converge in approximately 55% of cases. When both methods converge, estimation accuracy is comparable with negligible differences in relative error, R2 and, perplexity/likelihood. However, when CML fails, the NN-based approach still provides reliable estimates. In all cases, the NN estimator achieves substantial computational gains, reaching a median speedup of 12.6x and up to 34.2x for higher model orders. Overall, results demonstrate that gradient-descent NN optimization can provide a fast and efficient alternative for interpretable AR parameter estimation.