Predictivity and Utility of Neural Surrogates of Multiscale PDEs

TL;DR

This paper critically examines the limitations of neural surrogates for multi-scale PDEs, highlighting spectral bias and information loss, and discusses where they can be genuinely useful.

Contribution

It provides a detailed analysis of the fundamental limitations of neural surrogates in multi-scale PDEs and suggests directions for future research and application.

Findings

Neural surrogates under-resolve high-frequency content due to spectral bias.

Coarse-graining leads to irreversible information loss.

Medium-range weather prediction benefits from neural surrogates, unlike chaotic multi-scale scenarios.

Abstract

Scientific machine learning is increasingly being spoken of as universal emulators for classical numerical solvers for multi-scale partial differential equations, but most apparent successes can be explained by facts that also define their limits. Many successful benchmarks live on low-dimensional solution manifolds where any competent reduced model will interpolate well. More fundamentally, neural surrogates systematically under-resolve high-frequency content due to spectral bias, and coarse-graining compounds this problem through irreversible information loss. In many multi-scale problems, no architecture or training procedure can fully recover what the coarse representation discards. Two simple examples are used to characterize spectral bias, coarse-graining and error accumulation. We discuss why medium-range weather prediction on reanalysis data sits in a favorable sweet spot and why this will not generalize to genuinely chaotic multi-scale scenarios. We identify domains where neural surrogates offer genuine value, propose further research on neural-classical hybrids, and call for better reporting standards.