UltraSTF: Ultra-Compact Model for Large-Scale Spatio-Temporal Forecasting

TL;DR

UltraSTF is a highly compact spatio-temporal forecasting model that captures intra-period dynamics using an attention-based shape bank, achieving state-of-the-art results with minimal parameters.

Contribution

It introduces UltraSTF, a novel ultra-compact model combining cross-period forecasting with an attention-based shape bank for improved spatio-temporal predictions.

Findings

Achieves state-of-the-art performance on LargeST benchmark.

Uses less than 0.2% of parameters compared to second-best methods.

Significantly extends the Pareto frontier of model size and accuracy.

Abstract

Spatio-temporal data, prevalent in real-world applications such as traffic monitoring, financial transactions, and ride-share demands, represents a specialized case of multivariate time series characterized by high dimensionality. This high dimensionality necessitates computationally efficient models and benefits from applying univariate forecasting approaches through channel-independent strategies. SparseTSF, a recently proposed competitive univariate forecasting model, leverages periodicity to achieve compactness by focusing on cross-period dynamics, extending the Pareto frontier in terms of model size and predictive performance. However, it underperforms on spatio-temporal data due to limited capture of intra-period temporal dependencies. To address this limitation, we propose UltraSTF, which integrates a cross-period forecasting component with an ultra-compact shape bank component. Our model efficiently captures recurring patterns in time series using the attention mechanism of the shape bank component, significantly enhancing its capability to learn intra-period dynamics. UltraSTF achieves state-of-the-art performance on the LargeST benchmark while utilizing fewer than 0.2% of the parameters required by the second-best methods, thereby further extending the Pareto frontier of existing approaches.