TSMixer: An All-MLP Architecture for Time Series Forecasting

TL;DR

TSMixer is a simple yet effective MLP-based architecture for time series forecasting that matches or surpasses state-of-the-art models on various benchmarks, emphasizing the power of mixing operations along time and feature dimensions.

Contribution

The paper introduces TSMixer, a novel MLP-based model that efficiently captures complex time series dynamics through mixing operations, challenging the need for more complex architectures.

Findings

TSMixer performs comparably to specialized deep learning models on academic benchmarks.

On the large-scale M5 retail dataset, TSMixer outperforms existing state-of-the-art methods.

The architecture highlights the importance of cross-variate and auxiliary information in forecasting.

Abstract

Real-world time-series datasets are often multivariate with complex dynamics. To capture this complexity, high capacity architectures like recurrent- or attention-based sequential deep learning models have become popular. However, recent work demonstrates that simple univariate linear models can outperform such deep learning models on several commonly used academic benchmarks. Extending them, in this paper, we investigate the capabilities of linear models for time-series forecasting and present Time-Series Mixer (TSMixer), a novel architecture designed by stacking multi-layer perceptrons (MLPs). TSMixer is based on mixing operations along both the time and feature dimensions to extract information efficiently. On popular academic benchmarks, the simple-to-implement TSMixer is comparable to specialized state-of-the-art models that leverage the inductive biases of specific benchmarks. On the challenging and large scale M5 benchmark, a real-world retail dataset, TSMixer demonstrates superior performance compared to the state-of-the-art alternatives. Our results underline the importance of efficiently utilizing cross-variate and auxiliary information for improving the performance of time series forecasting. We present various analyses to shed light into the capabilities of TSMixer. The design paradigms utilized in TSMixer are expected to open new horizons for deep learning-based time series forecasting. The implementation is available at https://github.com/google-research/google-research/tree/master/tsmixer