Sparse Diffusion Autoencoder for Test-time Adapting Prediction of Complex Systems

TL;DR

SparseDiff introduces a test-time adaptive autoencoder that captures emergent spatiotemporal dynamics in complex systems, significantly improving long-term prediction accuracy with minimal spatial resolution.

Contribution

It proposes a novel sparse codebook-based encoder and graph neural ODE framework for dynamic adaptation during long-term predictions of complex systems.

Findings

Achieves 49.99% reduction in prediction error compared to baselines.

Requires only 1% of the original spatial resolution.

Effectively models emergent spatiotemporal patterns.

Abstract

Predicting the behavior of complex systems is critical in many scientific and engineering domains, and hinges on the model's ability to capture their underlying dynamics. Existing methods encode the intrinsic dynamics of high-dimensional observations through latent representations and predict autoregressively. However, these latent representations lose the inherent spatial structure of spatiotemporal dynamics, leading to the predictor's inability to effectively model spatial interactions and neglect emerging dynamics during long-term prediction. In this work, we propose SparseDiff, introducing a test-time adaptation strategy to dynamically update the encoding scheme to accommodate emergent spatiotemporal structures during the long-term evolution of the system. Specifically, we first design a codebook-based sparse encoder, which coarsens the continuous spatial domain into a sparse graph topology. Then, we employ a graph neural ordinary differential equation to model the dynamics and guide a diffusion decoder for reconstruction. SparseDiff autoregressively predicts the spatiotemporal evolution and adjust the sparse topological structure to adapt to emergent spatiotemporal patterns by adaptive re-encoding. Extensive evaluations on representative systems demonstrate that SparseDiff achieves an average prediction error reduction of 49.99\% compared to baselines, requiring only 1% of the spatial resolution.