TwinS: Revisiting Non-Stationarity in Multivariate Time Series Forecasting

TL;DR

This paper introduces TwinS, a Transformer-based model designed to handle the complex non-stationary characteristics of real-world multivariate time series, achieving state-of-the-art forecasting accuracy.

Contribution

The paper proposes a novel TwinS model with three modules—Wavelet Convolution, Period-Aware Attention, and Channel-Temporal MLP—to effectively model non-stationary periodic distributions in time series.

Findings

TwinS outperforms mainstream models with up to 25.8% MSE improvement.

Wavelet Convolution captures nested periodicity effectively.

Period-Aware Attention improves relevance scoring for periodic patterns.

Abstract

Recently, multivariate time series forecasting tasks have garnered increasing attention due to their significant practical applications, leading to the emergence of various deep forecasting models. However, real-world time series exhibit pronounced non-stationary distribution characteristics. These characteristics are not solely limited to time-varying statistical properties highlighted by non-stationary Transformer but also encompass three key aspects: nested periodicity, absence of periodic distributions, and hysteresis among time variables. In this paper, we begin by validating this theory through wavelet analysis and propose the Transformer-based TwinS model, which consists of three modules to address the non-stationary periodic distributions: Wavelet Convolution, Period-Aware Attention, and Channel-Temporal Mixed MLP. Specifically, The Wavelet Convolution models nested periods by scaling the convolution kernel size like wavelet transform. The Period-Aware Attention guides attention computation by generating period relevance scores through a convolutional sub-network. The Channel-Temporal Mixed MLP captures the overall relationships between time series through channel-time mixing learning. TwinS achieves SOTA performance compared to mainstream TS models, with a maximum improvement in MSE of 25.8\% over PatchTST.