Neural Emulator Superiority: When Machine Learning for PDEs Surpasses its Training Data

TL;DR

Neural emulators trained on low-fidelity PDE solver data can outperform the original high-fidelity solvers in accuracy, due to their ability to learn more regularized and error-resilient dynamics, challenging traditional assumptions about training data limitations.

Contribution

This paper introduces the concept of 'emulator superiority,' showing neural networks can surpass their training data's fidelity through theoretical analysis and empirical validation across PDEs.

Findings

Neural emulators can outperform high-fidelity solvers in accuracy.

Emulators implicitly learn more regularized dynamics.

Performance depends on training objectives and error characteristics.

Abstract

Neural operators or emulators for PDEs trained on data from numerical solvers are conventionally assumed to be limited by their training data's fidelity. We challenge this assumption by identifying "emulator superiority," where neural networks trained purely on low-fidelity solver data can achieve higher accuracy than those solvers when evaluated against a higher-fidelity reference. Our theoretical analysis reveals how the interplay between emulator inductive biases, training objectives, and numerical error characteristics enables superior performance during multi-step rollouts. We empirically validate this finding across different PDEs using standard neural architectures, demonstrating that emulators can implicitly learn dynamics that are more regularized or exhibit more favorable error accumulation properties than their training data, potentially surpassing training data limitations and mitigating numerical artifacts. This work prompts a re-evaluation of emulator benchmarking, suggesting neural emulators might achieve greater physical fidelity than their training source within specific operational regimes. Project Page: https://tum-pbs.github.io/emulator-superiority