Loss Landscape Geometry and the Learning of Symmetries: Or, What Influence Functions Reveal About Robust Generalization

TL;DR

This paper introduces an influence-based diagnostic to analyze how neural emulators of PDEs internalize physical symmetries, revealing the role of loss landscape geometry in symmetry generalization.

Contribution

It presents a new influence-based diagnostic that measures the propagation of parameter updates across symmetry-related states, linking loss landscape geometry to symmetry learning in neural PDE emulators.

Findings

Gradient coherence across symmetry orbits correlates with symmetry generalization.

The diagnostic can identify when training converges to symmetry-compatible basins.

The method provides a way to evaluate if models have internalized physical symmetries.

Abstract

We study how neural emulators of partial differential equation solution operators internalize physical symmetries by introducing an influence-based diagnostic that measures the propagation of parameter updates between symmetry-related states, defined as the metric-weighted overlap of loss gradients evaluated along group orbits. This quantity probes the local geometry of the learned loss landscape and goes beyond forward-pass equivariance tests by directly assessing whether learning dynamics couple physically equivalent configurations. Applying our diagnostic to autoregressive fluid flow emulators, we show that orbit-wise gradient coherence provides the mechanism for learning to generalize over symmetry transformations and indicates when training selects a symmetry compatible basin. The result is a novel technique for evaluating if surrogate models have internalized symmetry properties of the known solution operator.