Beyond Attention: Learning Spatio-Temporal Dynamics with Emergent Interpretable Topologies

TL;DR

This paper introduces InterGAT, a learnable, interpretable model for spatio-temporal forecasting that improves accuracy and efficiency over traditional GATs by capturing latent spatial relationships without fixed graph structures.

Contribution

It proposes InterGAT, a novel approach replacing attention with a learnable interaction matrix, enhancing interpretability, accuracy, and computational efficiency in spatio-temporal modeling.

Findings

Achieves at least 21% accuracy improvement on SZ-Taxi dataset.

Reduces training time by 60-70% compared to GAT-GRU.

Learned matrices reveal interpretable, community-aware topologies.

Abstract

Spatio-temporal forecasting is critical in applications such as traffic prediction, energy demand modeling, and weather monitoring. While Graph Attention Networks (GATs) are popular for modeling spatial dependencies, they rely on predefined adjacency structures and dynamic attention scores, introducing inductive biases and computational overhead that can obscure interpretability. We propose InterGAT, a simplified alternative to GAT that replaces masked attention with a fully learnable, symmetric node interaction matrix, capturing latent spatial relationships without relying on fixed graph topologies. Our framework, InterGAT-GRU, which incorporates a GRU-based temporal decoder, outperforms the baseline GAT-GRU in forecasting accuracy, achieving at least a 21% improvement on the SZ-Taxi dataset and a 6% improvement on the Los-Loop dataset across all forecasting horizons (15 to 60 minutes). Additionally, we observed reduction in training time by 60-70% compared to GAT-GRU baseline. Crucially, the learned interaction matrix reveals interpretable structure: it recovers sparse, topology-aware attention patterns that align with community structure. Spectral and clustering analyses show that the model captures both localized and global dynamics, offering insights into the functional topology driving predictions. This highlights how structure learning can simultaneously support prediction, computational efficiency, and topological interpretabil-ity in dynamic graph-based domains.