Forecaster: A Graph Transformer for Forecasting Spatial and Time-Dependent Data

TL;DR

Forecaster is a novel graph Transformer architecture designed to effectively model complex spatial and temporal dependencies in data, improving forecasting accuracy in applications like taxi demand prediction.

Contribution

The paper introduces a graph Transformer that learns spatial structure and sparsifies attention to handle non-stationarity and heterogeneity in spatiotemporal data.

Findings

Significantly outperforms state-of-the-art baselines in taxi demand forecasting.

Effectively models long-range temporal dependencies.

Adapts to data heterogeneity and non-stationarity.

Abstract

Spatial and time-dependent data is of interest in many applications. This task is difficult due to its complex spatial dependency, long-range temporal dependency, data non-stationarity, and data heterogeneity. To address these challenges, we propose Forecaster, a graph Transformer architecture. Specifically, we start by learning the structure of the graph that parsimoniously represents the spatial dependency between the data at different locations. Based on the topology of the graph, we sparsify the Transformer to account for the strength of spatial dependency, long-range temporal dependency, data non-stationarity, and data heterogeneity. We evaluate Forecaster in the problem of forecasting taxi ride-hailing demand and show that our proposed architecture significantly outperforms the state-of-the-art baselines.