ST-ReP: Learning Predictive Representations Efficiently for Spatial-Temporal Forecasting

TL;DR

ST-ReP is a lightweight self-supervised model that learns efficient, predictive spatial-temporal representations by integrating value reconstruction, future prediction, and multi-scale analysis, outperforming existing methods in scalability and accuracy.

Contribution

The paper introduces ST-ReP, a novel pre-training framework with a specialized encoder and multi-scale analysis for improved spatial-temporal forecasting.

Findings

Outperforms baseline models in diverse domains.

Learns compact, semantically rich representations.

Demonstrates superior scalability and predictive accuracy.

Abstract

Spatial-temporal forecasting is crucial and widely applicable in various domains such as traffic, energy, and climate. Benefiting from the abundance of unlabeled spatial-temporal data, self-supervised methods are increasingly adapted to learn spatial-temporal representations. However, it encounters three key challenges: 1) the difficulty in selecting reliable negative pairs due to the homogeneity of variables, hindering contrastive learning methods; 2) overlooking spatial correlations across variables over time; 3) limitations of efficiency and scalability in existing self-supervised learning methods. To tackle these, we propose a lightweight representation-learning model ST-ReP, integrating current value reconstruction and future value prediction into the pre-training framework for spatial-temporal forecasting. And we design a new spatial-temporal encoder to model fine-grained relationships. Moreover, multi-time scale analysis is incorporated into the self-supervised loss to enhance predictive capability. Experimental results across diverse domains demonstrate that the proposed model surpasses pre-training-based baselines, showcasing its ability to learn compact and semantically enriched representations while exhibiting superior scalability.