Weaver: Kronecker Product Approximations of Spatiotemporal Attention for Traffic Network Forecasting

TL;DR

Weaver is an efficient, interpretable attention-based model for traffic network forecasting that uses Kronecker product approximations to reduce complexity and incorporates novel mechanisms for modeling traffic dynamics.

Contribution

We introduce Weaver, a novel spatiotemporal attention model using Kronecker product approximations and new attention mechanisms for traffic forecasting.

Findings

Achieves competitive accuracy on PEMS-BAY and METR-LA datasets.

Reduces computational complexity compared to traditional Transformer models.

Demonstrates improved training efficiency and interpretability.

Abstract

Spatiotemporal forecasting on transportation networks is a complex task that requires understanding how traffic nodes interact within a dynamic, evolving system dictated by traffic flow dynamics and social behavioral patterns. The importance of transportation networks and ITS for modern mobility and commerce necessitates forecasting models that are not only accurate but also interpretable, efficient, and robust under structural or temporal perturbations. Recent approaches, particularly Transformer-based architectures, have improved predictive performance but often at the cost of high computational overhead and diminished architectural interpretability. In this work, we introduce Weaver, a novel attention-based model that applies Kronecker product approximations (KPA) to decompose the PN X PN spatiotemporal attention of O(P^2N^2) complexity into local P X P temporal and N X N spatial attention maps. This Kronecker attention map enables our Parallel-Kronecker Matrix-Vector product (P2-KMV) for efficient spatiotemporal message passing with O(P^2N + N^2P) complexity. To capture real-world traffic dynamics, we address the importance of negative edges in modeling traffic behavior by introducing Valence Attention using the continuous Tanimoto coefficient (CTC), which provides properties conducive to precise latent graph generation and training stability. To fully utilize the model's learning capacity, we introduce the Traffic Phase Dictionary for self-conditioning. Evaluations on PEMS-BAY and METR-LA show that Weaver achieves competitive performance across model categories while training more efficiently.