Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting

TL;DR

This paper introduces a lightweight prompt tuning approach for continual spatio-temporal graph forecasting, addressing challenges of model expansion and forgetting in streaming data scenarios.

Contribution

It proposes the expand and compress principles for tuning, integrating a prompt pool with a base graph neural network for efficient continual learning.

Findings

Outperforms state-of-the-art methods in accuracy and efficiency

Effectively mitigates catastrophic forgetting in streaming data

Demonstrates universality across multiple datasets

Abstract

The widespread deployment of sensing devices leads to a surge in data for spatio-temporal forecasting applications such as traffic flow, air quality, and wind energy. Although spatio-temporal graph neural networks have achieved success in modeling various static spatio-temporal forecasting scenarios, real-world spatio-temporal data are typically received in a streaming manner, and the network continuously expands with the installation of new sensors. Thus, spatio-temporal forecasting in streaming scenarios faces dual challenges: the inefficiency of retraining models over newly arrived data and the detrimental effects of catastrophic forgetting over long-term history. To address these challenges, we propose a novel prompt tuning-based continuous forecasting method, following two fundamental tuning principles guided by empirical and theoretical analysis: expand and compress, which effectively resolve the aforementioned problems with lightweight tuning parameters. Specifically, we integrate the base spatio-temporal graph neural network with a continuous prompt pool, utilizing stored prompts (i.e., few learnable parameters) in memory, and jointly optimize them with the base spatio-temporal graph neural network. This method ensures that the model sequentially learns from the spatio-temporal data stream to accomplish tasks for corresponding periods. Extensive experimental results on multiple real-world datasets demonstrate the multi-faceted superiority of our method over the state-of-the-art baselines, including effectiveness, efficiency, universality, etc.