ADFormer: Aggregation Differential Transformer for Passenger Demand Forecasting

TL;DR

ADFormer introduces a novel transformer model that effectively captures both original and high-level spatio-temporal correlations for improved passenger demand forecasting, demonstrating superior performance on real-world datasets.

Contribution

The paper proposes Differential Attention and aggregation strategies to unify original and high-level correlations, enhancing spatio-temporal modeling in demand forecasting.

Findings

Outperforms existing methods on taxi and bike datasets

Demonstrates improved accuracy and efficiency

Validates the importance of high-level correlation integration

Abstract

Passenger demand forecasting helps optimize vehicle scheduling, thereby improving urban efficiency. Recently, attention-based methods have been used to adequately capture the dynamic nature of spatio-temporal data. However, existing methods that rely on heuristic masking strategies cannot fully adapt to the complex spatio-temporal correlations, hindering the model from focusing on the right context. These works also overlook the high-level correlations that exist in the real world. Effectively integrating these high-level correlations with the original correlations is crucial. To fill this gap, we propose the Aggregation Differential Transformer (ADFormer), which offers new insights to demand forecasting promotion. Specifically, we utilize Differential Attention to capture the original spatial correlations and achieve attention denoising. Meanwhile, we design distinct aggregation strategies based on the nature of space and time. Then, the original correlations are unified with the high-level correlations, enabling the model to capture holistic spatio-temporal relations. Experiments conducted on taxi and bike datasets confirm the effectiveness and efficiency of our model, demonstrating its practical value. The code is available at https://github.com/decisionintelligence/ADFormer.