Towards Infinitely Long Neural Simulations: Self-Refining Neural Surrogate Models for Dynamical Systems

TL;DR

This paper introduces SNS, a hyperparameter-free diffusion model that improves long-term accuracy in neural simulations of dynamical systems by self-refinement, addressing the long-standing issue of distribution drift in autoregressive models.

Contribution

The paper formalizes the accuracy-consistency tradeoff in neural surrogates and proposes SNS, a novel self-refining diffusion model that ensures long-term simulation fidelity without hyperparameter tuning.

Findings

SNS effectively maintains long-term accuracy in complex dynamical systems.

The model can be used standalone or with existing surrogates for improved consistency.

Numerical experiments demonstrate high-fidelity long-horizon simulations.

Abstract

Recent advances in autoregressive neural surrogate models have enabled orders-of-magnitude speedups in simulating dynamical systems. However, autoregressive models are generally prone to distribution drift: compounding errors in autoregressive rollouts that severely degrade generation quality over long time horizons. Existing work attempts to address this issue by implicitly leveraging the inherent trade-off between short-time accuracy and long-time consistency through hyperparameter tuning. In this work, we introduce a unifying mathematical framework that makes this tradeoff explicit, formalizing and generalizing hyperparameter-based strategies in existing approaches. Within this framework, we propose a robust, hyperparameter-free model implemented as a conditional diffusion model that balances short-time fidelity with long-time consistency by construction. Our model, Self-refining Neural Surrogate model (SNS), can be implemented as a standalone model that refines its own autoregressive outputs or as a complementary model to existing neural surrogates to ensure long-time consistency. We also demonstrate the numerical feasibility of SNS through high-fidelity simulations of complex dynamical systems over arbitrarily long time horizons.