Efficient Traffic Prediction Through Spatio-Temporal Distillation

TL;DR

This paper introduces LightST, a knowledge distillation framework that enables lightweight models to efficiently predict traffic flow by capturing global spatio-temporal patterns, significantly improving speed while maintaining accuracy.

Contribution

The paper proposes a novel spatio-temporal knowledge distillation method, LightST, to address GNN scalability and over-smoothing issues in traffic prediction.

Findings

LightST speeds up predictions by 5X to 40X.

LightST maintains superior accuracy compared to state-of-the-art GNNs.

The framework effectively captures global spatio-temporal patterns.

Abstract

Graph neural networks (GNNs) have gained considerable attention in recent years for traffic flow prediction due to their ability to learn spatio-temporal pattern representations through a graph-based message-passing framework. Although GNNs have shown great promise in handling traffic datasets, their deployment in real-life applications has been hindered by scalability constraints arising from high-order message passing. Additionally, the over-smoothing problem of GNNs may lead to indistinguishable region representations as the number of layers increases, resulting in performance degradation. To address these challenges, we propose a new knowledge distillation paradigm termed LightST that transfers spatial and temporal knowledge from a high-capacity teacher to a lightweight student. Specifically, we introduce a spatio-temporal knowledge distillation framework that helps student MLPs capture graph-structured global spatio-temporal patterns while alleviating the over-smoothing effect with adaptive knowledge distillation. Extensive experiments verify that LightST significantly speeds up traffic flow predictions by 5X to 40X compared to state-of-the-art spatio-temporal GNNs, all while maintaining superior accuracy.