Diagnosing Failure Modes of Neural Operators Across Diverse PDE Families

TL;DR

This paper introduces a standardized stress-testing framework to evaluate the robustness of neural PDE solvers across diverse PDE families and architectures, highlighting the gap between in-distribution accuracy and robustness.

Contribution

It presents a comprehensive evaluation method for neural PDE solvers under deployment-relevant shifts, revealing architecture and PDE family-dependent failure patterns.

Findings

Strong in-distribution accuracy does not predict robustness.

Failure patterns depend on architecture and PDE family.

Spectral and rollout diagnostics help identify robustness issues.

Abstract

Neural PDE solvers are increasingly used as learned surrogates for families of partial differential equations, where the key machine learning challenge is not only interpolation on a fixed benchmark distribution but generalization under structured shifts in coefficients, boundary conditions, discretization, and rollout horizon. Yet evaluation is still often dominated by in-distribution test error, making robustness difficult to assess. We introduce a standardized stress-testing framework for neural PDE solvers under deployment-relevant shift. We instantiate it on three representative architectures -- Fourier Neural Operators (FNOs), a DeepONet-style model, and convolutional neural operators (CNOs) -- across five qualitatively different PDE families: dispersive, elliptic, multi-scale fluid, financial, and chaotic systems. Across 750 trained models, we measure robustness using baseline-normalized degradation factors together with spectral and rollout diagnostics. The resulting comparisons reveal that strong in-distribution accuracy does not reliably predict robustness, and that failure patterns depend jointly on architecture and PDE family. Our results provide a clearer basis for evaluating robustness claims in neural PDE solvers and suggest that function-space generalization under structured shift should be treated as a first-class evaluation target.