FaST: Efficient and Effective Long-Horizon Forecasting for Large-Scale Spatial-Temporal Graphs via Mixture-of-Experts

TL;DR

FaST introduces a scalable, efficient framework for long-horizon spatial-temporal graph forecasting using heterogeneity-aware Mixture-of-Experts, enabling accurate predictions over large networks with reduced computational costs.

Contribution

The paper proposes a novel heterogeneity-aware Mixture-of-Experts framework with adaptive graph attention and parallel MoE modules for scalable long-horizon STG forecasting.

Findings

Outperforms state-of-the-art models in accuracy for long-horizon predictions.

Achieves significant reductions in computational time and memory usage.

Successfully predicts one-week-ahead traffic patterns on large-scale graphs.

Abstract

Spatial-Temporal Graph (STG) forecasting on large-scale networks has garnered significant attention. However, existing models predominantly focus on short-horizon predictions and suffer from notorious computational costs and memory consumption when scaling to long-horizon predictions and large graphs. Targeting the above challenges, we present FaST, an effective and efficient framework based on heterogeneity-aware Mixture-of-Experts (MoEs) for long-horizon and large-scale STG forecasting, which unlocks one-week-ahead (672 steps at a 15-minute granularity) prediction with thousands of nodes. FaST is underpinned by two key innovations. First, an adaptive graph agent attention mechanism is proposed to alleviate the computational burden inherent in conventional graph convolution and self-attention modules when applied to large-scale graphs. Second, we propose a new parallel MoE module that replaces traditional feed-forward networks with Gated Linear Units (GLUs), enabling an efficient and scalable parallel structure. Extensive experiments on real-world datasets demonstrate that FaST not only delivers superior long-horizon predictive accuracy but also achieves remarkable computational efficiency compared to state-of-the-art baselines. Our source code is available at: https://github.com/yijizhao/FaST.