Interpretability and Generalization Bounds for Learning Spatial Physics

TL;DR

This paper rigorously analyzes the accuracy, convergence, and generalization of ML models for linear differential equations in physics, revealing the importance of data function space and introducing interpretability and validation techniques.

Contribution

It provides theoretical bounds, empirical insights, and a new interpretability framework for physics-informed ML models, along with a novel cross-validation method for physical systems.

Findings

Function space critically affects model generalization.

Different model classes show opposing generalization behaviors.

Green's function can be extracted from black-box model weights.

Abstract

While there are many applications of ML to scientific problems that look promising, visuals can be deceiving. Using numerical analysis techniques, we rigorously quantify the accuracy, convergence rates, and generalization bounds of certain ML models applied to linear differential equations for parameter discovery or solution finding. Beyond the quantity and discretization of data, we identify that the function space of the data is critical to the generalization of the model. A similar lack of generalization is empirically demonstrated for commonly used models, including physics-specific techniques. Counterintuitively, we find that different classes of models can exhibit opposing generalization behaviors. Based on our theoretical analysis, we also introduce a new mechanistic interpretability lens on scientific models whereby Green's function representations can be extracted from the weights of black-box models. Our results inform a new cross-validation technique for measuring generalization in physical systems, which can serve as a benchmark.