Spatiotemporal System Forecasting with Irregular Time Steps via Masked Autoencoder

TL;DR

This paper introduces a Physics-Spatiotemporal Masked Autoencoder that effectively predicts high-dimensional dynamical systems with irregular time steps, improving accuracy and efficiency without data imputation.

Contribution

The novel model combines convolutional and masked autoencoders with attention mechanisms to handle irregular time series in high-dimensional systems, preserving physical integrity.

Findings

Significant accuracy improvements over traditional methods

Robustness to nonlinear and irregular data

Enhanced computational efficiency

Abstract

Predicting high-dimensional dynamical systems with irregular time steps presents significant challenges for current data-driven algorithms. These irregularities arise from missing data, sparse observations, or adaptive computational techniques, reducing prediction accuracy. To address these limitations, we propose a novel method: a Physics-Spatiotemporal Masked Autoencoder. This method integrates convolutional autoencoders for spatial feature extraction with masked autoencoders optimised for irregular time series, leveraging attention mechanisms to reconstruct the entire physical sequence in a single prediction pass. The model avoids the need for data imputation while preserving physical integrity of the system. Here, 'physics' refers to high-dimensional fields generated by underlying dynamical systems, rather than the enforcement of explicit physical constraints or PDE residuals. We evaluate this approach on multiple simulated datasets and real-world ocean temperature data. The results demonstrate that our method achieves significant improvements in prediction accuracy, robustness to nonlinearities, and computational efficiency over traditional convolutional and recurrent network methods. The model shows potential for capturing complex spatiotemporal patterns without requiring domain-specific knowledge, with applications in climate modelling, fluid dynamics, ocean forecasting, environmental monitoring, and scientific computing.