Considering Nonstationary within Multivariate Time Series with Variational Hierarchical Transformer for Forecasting

TL;DR

This paper introduces HTV-Trans, a hierarchical variational transformer model that effectively captures non-stationary and stochastic features in multivariate time series for improved forecasting accuracy.

Contribution

It proposes a novel hierarchical probabilistic generative module combined with transformer architecture to model non-stationarity and stochasticity in MTS, advancing forecasting methods.

Findings

HTV-Trans outperforms existing models on diverse datasets.

The model effectively captures non-stationary and stochastic features.

Extensive experiments demonstrate its forecasting efficiency.

Abstract

The forecasting of Multivariate Time Series (MTS) has long been an important but challenging task. Due to the non-stationary problem across long-distance time steps, previous studies primarily adopt stationarization method to attenuate the non-stationary problem of the original series for better predictability. However, existing methods always adopt the stationarized series, which ignores the inherent non-stationarity, and has difficulty in modeling MTS with complex distributions due to the lack of stochasticity. To tackle these problems, we first develop a powerful hierarchical probabilistic generative module to consider the non-stationarity and stochastic characteristics within MTS, and then combine it with transformer for a well-defined variational generative dynamic model named Hierarchical Time series Variational Transformer (HTV-Trans), which recovers the intrinsic non-stationary information into temporal dependencies. Being a powerful probabilistic model, HTV-Trans is utilized to learn expressive representations of MTS and applied to forecasting tasks. Extensive experiments on diverse datasets show the efficiency of HTV-Trans on MTS forecasting tasks