DeformTime: Capturing Variable Dependencies with Deformable Attention for Time Series Forecasting

TL;DR

DeformTime is a novel neural network architecture employing deformable attention to better capture variable and temporal dependencies in multivariable time series forecasting, leading to improved accuracy especially with exogenous variables.

Contribution

It introduces deformable attention blocks to explicitly model variable and temporal dependencies, enhancing forecasting performance over existing methods.

Findings

Achieves 7.2% average reduction in mean absolute error.

Consistent performance improvements across longer forecasting horizons.

Effective on diverse datasets including infectious disease modeling.

Abstract

In multivariable time series (MTS) forecasting, existing state-of-the-art deep learning approaches tend to focus on autoregressive formulations and often overlook the potential of using exogenous variables in enhancing the prediction of the target endogenous variable. To address this limitation, we present DeformTime, a neural network architecture that attempts to capture correlated temporal patterns from the input space, and hence, improve forecasting accuracy. It deploys two core operations performed by deformable attention blocks (DABs): learning dependencies across variables from different time steps (variable DAB), and preserving temporal dependencies in data from previous time steps (temporal DAB). Input data transformation is explicitly designed to enhance learning from the deformed series of information while passing through a DAB. We conduct extensive experiments on 6 MTS data sets, using previously established benchmarks as well as challenging infectious disease modelling tasks with more exogenous variables. The results demonstrate that DeformTime improves accuracy against previous competitive methods across the vast majority of MTS forecasting tasks, reducing the mean absolute error by 7.2% on average. Notably, performance gains remain consistent across longer forecasting horizons.