Localised Adaptive Spatial-Temporal Graph Neural Network

TL;DR

This paper demonstrates that spatial dependencies in adaptive spatial-temporal graph neural networks can be almost entirely localised or removed without loss of inference accuracy, potentially reducing computational costs.

Contribution

The authors introduce Adaptive Graph Sparsification (AGS), enabling extreme localisation of ASTGNNs, and show that spatial graphs can be sparsified by over 99.5% without accuracy loss.

Findings

Spatial graphs in ASTGNNs can be sparsified by over 99.5% without accuracy decline.

Fully localised ASTGNNs, with no spatial graph, maintain accuracy on most datasets.

Reinitialising and retraining localised ASTGNNs causes a significant accuracy drop.

Abstract

Spatial-temporal graph models are prevailing for abstracting and modelling spatial and temporal dependencies. In this work, we ask the following question: whether and to what extent can we localise spatial-temporal graph models? We limit our scope to adaptive spatial-temporal graph neural networks (ASTGNNs), the state-of-the-art model architecture. Our approach to localisation involves sparsifying the spatial graph adjacency matrices. To this end, we propose Adaptive Graph Sparsification (AGS), a graph sparsification algorithm which successfully enables the localisation of ASTGNNs to an extreme extent (fully localisation). We apply AGS to two distinct ASTGNN architectures and nine spatial-temporal datasets. Intriguingly, we observe that spatial graphs in ASTGNNs can be sparsified by over 99.5\% without any decline in test accuracy. Furthermore, even when ASTGNNs are fully localised, becoming graph-less and purely temporal, we record no drop in accuracy for the majority of tested datasets, with only minor accuracy deterioration observed in the remaining datasets. However, when the partially or fully localised ASTGNNs are reinitialised and retrained on the same data, there is a considerable and consistent drop in accuracy. Based on these observations, we reckon that \textit{(i)} in the tested data, the information provided by the spatial dependencies is primarily included in the information provided by the temporal dependencies and, thus, can be essentially ignored for inference; and \textit{(ii)} although the spatial dependencies provide redundant information, it is vital for the effective training of ASTGNNs and thus cannot be ignored during training. Furthermore, the localisation of ASTGNNs holds the potential to reduce the heavy computation overhead required on large-scale spatial-temporal data and further enable the distributed deployment of ASTGNNs.