Transformers \`a Grande Vitesse

TL;DR

This paper introduces a transformer-based method for real-time train delay prediction across entire rail networks, capturing delay propagation phenomena and outperforming existing techniques.

Contribution

It presents a novel transformer architecture with pre-trained embeddings for large-scale, real-time train delay forecasting, addressing complex network interactions.

Findings

Effective delay prediction for over 3000 trains during peak hours

Captures delay propagation phenomena in railway networks

Outperforms existing prediction methods

Abstract

Robust travel time predictions are of prime importance in managing any transportation infrastructure, and particularly in rail networks where they have major impacts both on traffic regulation and passenger satisfaction. We aim at predicting the travel time of trains on rail sections at the scale of an entire rail network in real-time, by estimating trains' delays relative to a theoretical circulation plan. Predicting the evolution of a given train's delay is a uniquely hard problem, distinct from mainstream road traffic forecasting problems, since it involves several hard-to-model phenomena: train spacing, station congestion and heterogeneous rolling stock among others. We first offer empirical evidence of the previously unexplored phenomenon of delay propagation at the scale of a railway network, leading to delays being amplified by interactions between trains and the network's physical limitations. We then contribute a novel technique using the transformer architecture and pre-trained embeddings to make real-time massively parallel predictions for train delays at the scale of the whole rail network (over 3000 trains at peak hours, making predictions at an average horizon of 70 minutes). Our approach yields very positive results on real-world data when compared to currently-used and experimental prediction techniques.