ETSformer: Exponential Smoothing Transformers for Time-series Forecasting

TL;DR

ETSFormer introduces a novel Transformer architecture for time-series forecasting that incorporates exponential smoothing principles, enhancing interpretability, decomposition capabilities, and efficiency for long-term predictions.

Contribution

The paper proposes ETSFormer, a Transformer model utilizing exponential smoothing attention and frequency attention to improve time-series decomposition and forecasting accuracy.

Findings

Outperforms existing methods on multiple benchmarks

Demonstrates improved interpretability and decomposition

Achieves better long-term forecasting accuracy

Abstract

Transformers have been actively studied for time-series forecasting in recent years. While often showing promising results in various scenarios, traditional Transformers are not designed to fully exploit the characteristics of time-series data and thus suffer some fundamental limitations, e.g., they generally lack of decomposition capability and interpretability, and are neither effective nor efficient for long-term forecasting. In this paper, we propose ETSFormer, a novel time-series Transformer architecture, which exploits the principle of exponential smoothing in improving Transformers for time-series forecasting. In particular, inspired by the classical exponential smoothing methods in time-series forecasting, we propose the novel exponential smoothing attention (ESA) and frequency attention (FA) to replace the self-attention mechanism in vanilla Transformers, thus improving both accuracy and efficiency. Based on these, we redesign the Transformer architecture with modular decomposition blocks such that it can learn to decompose the time-series data into interpretable time-series components such as level, growth and seasonality. Extensive experiments on various time-series benchmarks validate the efficacy and advantages of the proposed method. Code is available at https://github.com/salesforce/ETSformer.