Time Series Forecasting Through the Lens of Dynamics

TL;DR

This paper introduces a dynamics-focused analysis of time series forecasting models, revealing how the placement of dynamics modules affects performance and proposing a plug-and-play design methodology.

Contribution

It develops the PRO-DYN framework to analyze models' learning dynamics and demonstrates the importance of dynamics block placement for improved forecasting.

Findings

Under-performing models learn dynamics only partially.

Placement of the dynamics block at the model's end is crucial.

A simple plug-and-play methodology can guide model improvements.

Abstract

While deep learning is facing an homogenization across modalities led by Transformers, they are still challenged by shallow linear models in the time series forecasting task. Our hypothesis is that models should learn a direct link from past to future data points, which we identify as a learning dynamics capability. We develop an original $\texttt{PRO-DYN}$ nomenclature to analyze existing models through the lens of dynamics. Two observations thus emerge: $\textbf{1.}$ under-performing architectures learn dynamics at most partially, $\textbf{2.}$ the location of the dynamics block at the model end is of prime importance. Our systemic and empirical studies both confirm our observations on a set of performance-varying models with diverse backbones. We propose a simple plug-and-play methodology guiding model designs and improvements.