A Pretrained Probabilistic Transformer for City-Scale Traffic Volume Prediction

TL;DR

This paper introduces TrafficPPT, a probabilistic transformer model for city-scale traffic prediction that models uncertainty, integrates diverse data sources, and demonstrates superior performance and adaptability across urban environments.

Contribution

The paper presents a pretrained probabilistic transformer that models traffic as a distribution, combining heterogeneous data and enabling domain adaptation for scalable city traffic prediction.

Findings

Outperforms state-of-the-art baselines in real-world datasets.

Effectively handles data sparsity and uncertainty.

Shows strong generalization across different cities.

Abstract

City-scale traffic volume prediction plays a pivotal role in intelligent transportation systems, yet remains a challenge due to the inherent incompleteness and bias in observational data. Although deep learning-based methods have shown considerable promise, most existing approaches produce deterministic point estimates, thereby neglecting the uncertainty arising from unobserved traffic flows. Furthermore, current models are typically trained in a city-specific manner, which hinders their generalizability and limits scalability across diverse urban contexts. To overcome these limitations, we introduce TrafficPPT, a Pretrained Probabilistic Transformer designed to model traffic volume as a distributional aggregation of trajectories. Our framework fuses heterogeneous data sources-including real-time observations, historical trajectory data, and road network topology-enabling robust and uncertainty-aware traffic inference. TrafficPPT is initially pretrained on large-scale simulated data spanning multiple urban scenarios, and later fine-tuned on target cities to ensure effective domain adaptation. Experiments on real-world datasets show that TrafficPPT consistently surpasses state-of-the-art baselines, particularly under conditions of extreme data sparsity. Code will be open.