AROpt: An Optimization Method for Autoregressive Time Series Forecasting

TL;DR

AROpt introduces a novel training approach for autoregressive time series forecasting that enforces error growth and enables flexible long-term predictions, achieving state-of-the-art results.

Contribution

The paper proposes a new training method that enforces error growth and concatenation of AR predictions, improving long-term forecasting accuracy.

Findings

Achieves over 10% MSE reduction on benchmarks.

Enables reliable long-term forecasts over 7.5 times longer horizons.

Sets new state-of-the-art performance in time-series forecasting.

Abstract

Current time-series forecasting models are primarily based on transformer-style neural networks. These models achieve long-term forecasting mainly by scaling up the model size rather than through genuinely autoregressive (AR) rollout. From the perspective of large language model training, traditional time-series forecasting model training ignores the monotonic error-growth heuristic. In this paper, we propose a novel training method for time-series forecasting that enforces two key properties: (1) AR prediction errors should increase with the forecasting horizon. Violations of this trend are interpreted as rollout inconsistency and are softly penalized during training, and (2) the method enables models to be able to concatenate short-term AR predictions to form flexible long-term forecasts. Empirical results demonstrate that our method establishes a new state-of-the-art across multiple benchmarks, achieving an MSE reduction of more than $10\%$ compared to iTransformer and other recent strong baselines. Furthermore, it enables short-horizon forecasting models to perform reliable long-term predictions at horizons over 7.5 times longer. Code is available at https://github.com/LizhengMathAi/AROpt