Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting

TL;DR

The paper introduces the Temporal Fusion Transformer (TFT), an attention-based neural network architecture that achieves high-accuracy multi-horizon time series forecasting while providing interpretability of temporal dynamics and feature relevance.

Contribution

The novel TFT architecture combines recurrent and attention mechanisms with feature selection and gating for improved, interpretable multi-horizon forecasting.

Findings

Significant performance improvements over benchmarks

Effective feature selection and interpretability demonstrations

Versatile across various real-world datasets

Abstract

Multi-horizon forecasting problems often contain a complex mix of inputs -- including static (i.e. time-invariant) covariates, known future inputs, and other exogenous time series that are only observed historically -- without any prior information on how they interact with the target. While several deep learning models have been proposed for multi-step prediction, they typically comprise black-box models which do not account for the full range of inputs present in common scenarios. In this paper, we introduce the Temporal Fusion Transformer (TFT) -- a novel attention-based architecture which combines high-performance multi-horizon forecasting with interpretable insights into temporal dynamics. To learn temporal relationships at different scales, the TFT utilizes recurrent layers for local processing and interpretable self-attention layers for learning long-term dependencies. The TFT also uses specialized components for the judicious selection of relevant features and a series of gating layers to suppress unnecessary components, enabling high performance in a wide range of regimes. On a variety of real-world datasets, we demonstrate significant performance improvements over existing benchmarks, and showcase three practical interpretability use-cases of TFT.