ExoTST: Exogenous-Aware Temporal Sequence Transformer for Time Series Prediction

TL;DR

ExoTST is a transformer-based framework that effectively integrates past and current exogenous variables with historical data for improved and robust long-term time series prediction, outperforming existing methods.

Contribution

The paper introduces ExoTST, a novel transformer model with a cross-temporal fusion module that jointly learns from past and current exogenous variables, addressing data uncertainties and distribution shifts.

Findings

Achieves up to 10% improvement in prediction accuracy over baselines.

Demonstrates robustness against missing values and noise in exogenous data.

Excels on real-world carbon flux datasets and benchmark time series tasks.

Abstract

Accurate long-term predictions are the foundations for many machine learning applications and decision-making processes. Traditional time series approaches for prediction often focus on either autoregressive modeling, which relies solely on past observations of the target ``endogenous variables'', or forward modeling, which considers only current covariate drivers ``exogenous variables''. However, effectively integrating past endogenous and past exogenous with current exogenous variables remains a significant challenge. In this paper, we propose ExoTST, a novel transformer-based framework that effectively incorporates current exogenous variables alongside past context for improved time series prediction. To integrate exogenous information efficiently, ExoTST leverages the strengths of attention mechanisms and introduces a novel cross-temporal modality fusion module. This module enables the model to jointly learn from both past and current exogenous series, treating them as distinct modalities. By considering these series separately, ExoTST provides robustness and flexibility in handling data uncertainties that arise from the inherent distribution shift between historical and current exogenous variables. Extensive experiments on real-world carbon flux datasets and time series benchmarks demonstrate ExoTST's superior performance compared to state-of-the-art baselines, with improvements of up to 10\% in prediction accuracy. Moreover, ExoTST exhibits strong robustness against missing values and noise in exogenous drivers, maintaining consistent performance in real-world situations where these imperfections are common.