TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts

TL;DR

TESTAM is a novel deep learning model that effectively captures complex spatio-temporal traffic patterns by using a mixture-of-experts approach, improving traffic forecasting accuracy on multiple datasets.

Contribution

The paper introduces TESTAM, a new model that separately models recurring and non-recurring traffic patterns with three specialized experts, enhancing spatio-temporal traffic prediction.

Findings

TESTAM outperforms existing models on METR-LA, PEMS-BAY, and EXPY-TKY datasets.

The mixture-of-experts approach improves modeling of diverse traffic scenarios.

Experimental results show better accuracy in capturing traffic dynamics.

Abstract

Accurate traffic forecasting is challenging due to the complex dependency on road networks, various types of roads, and the abrupt speed change due to the events. Recent works mainly focus on dynamic spatial modeling with adaptive graph embedding or graph attention having less consideration for temporal characteristics and in-situ modeling. In this paper, we propose a novel deep learning model named TESTAM, which individually models recurring and non-recurring traffic patterns by a mixture-of-experts model with three experts on temporal modeling, spatio-temporal modeling with static graph, and dynamic spatio-temporal dependency modeling with dynamic graph. By introducing different experts and properly routing them, TESTAM could better model various circumstances, including spatially isolated nodes, highly related nodes, and recurring and non-recurring events. For the proper routing, we reformulate a gating problem into a classification problem with pseudo labels. Experimental results on three public traffic network datasets, METR-LA, PEMS-BAY, and EXPY-TKY, demonstrate that TESTAM achieves a better indication and modeling of recurring and non-recurring traffic. We published the official code at https://github.com/HyunWookL/TESTAM