Enhanced accuracy through ensembling of randomly initialized auto-regressive models for time-dependent PDEs

TL;DR

This paper introduces a deep ensemble approach with randomly initialized auto-regressive models to improve long-term prediction accuracy of PDE systems, reducing error accumulation and enabling faster inference than traditional numerical methods.

Contribution

The paper presents a novel ensemble framework that enhances autoregressive ML models for PDEs, significantly improving long-term accuracy and computational efficiency.

Findings

Consistent reduction in error over time across three PDE systems.

Enables full trajectory predictions with minimal initial data.

Inference times are significantly faster than traditional solvers.

Abstract

Systems governed by partial differential equations (PDEs) require computationally intensive numerical solvers to predict spatiotemporal field evolution. While machine learning (ML) surrogates offer faster solutions, autoregressive inference with ML models suffer from error accumulation over successive predictions, limiting their long-term accuracy. We propose a deep ensemble framework to address this challenge, where multiple ML surrogate models with random weight initializations are trained in parallel and aggregated during inference. This approach leverages the diversity of model predictions to mitigate error propagation while retaining the autoregressive strategies ability to capture the system's time dependent relations. We validate the framework on three PDE-driven dynamical systems - stress evolution in heterogeneous microstructures, Gray-Scott reaction-diffusion, and planetary-scale shallow water system - demonstrating consistent reduction in error accumulation over time compared to individual models. Critically, the method requires only a few time steps as input, enabling full trajectory predictions with inference times significantly faster than numerical solvers. Our results highlight the robustness of ensemble methods in diverse physical systems and their potential as efficient and accurate alternatives to traditional solvers. The codes for this work are available on GitHub (https://github.com/Graham-Brady-Research-Group/AutoregressiveEnsemble_SpatioTemporal_Evolution).