MTS-Mixers: Multivariate Time Series Forecasting via Factorized Temporal and Channel Mixing

TL;DR

This paper introduces MTS-Mixers, a novel model for multivariate time series forecasting that effectively captures temporal and channel dependencies without relying on attention mechanisms, outperforming Transformer-based models in efficiency.

Contribution

The paper proposes MTS-Mixers, a new approach that uses factorized modules to model temporal and channel interactions, challenging the necessity of attention in forecasting tasks.

Findings

MTS-Mixers outperform existing Transformer-based models in accuracy.

Attention mechanisms are not essential for capturing temporal dependencies.

MTS-Mixers achieve higher efficiency in real-world datasets.

Abstract

Multivariate time series forecasting has been widely used in various practical scenarios. Recently, Transformer-based models have shown significant potential in forecasting tasks due to the capture of long-range dependencies. However, recent studies in the vision and NLP fields show that the role of attention modules is not clear, which can be replaced by other token aggregation operations. This paper investigates the contributions and deficiencies of attention mechanisms on the performance of time series forecasting. Specifically, we find that (1) attention is not necessary for capturing temporal dependencies, (2) the entanglement and redundancy in the capture of temporal and channel interaction affect the forecasting performance, and (3) it is important to model the mapping between the input and the prediction sequence. To this end, we propose MTS-Mixers, which use two factorized modules to capture temporal and channel dependencies. Experimental results on several real-world datasets show that MTS-Mixers outperform existing Transformer-based models with higher efficiency.