Understanding Generalization in Physics Informed Models through Affine Variety Dimensions

TL;DR

This paper investigates how the generalization ability of physics-informed models depends on the affine variety dimension of physical constraints, offering a unified framework for linear and nonlinear systems with incomplete data.

Contribution

It introduces a unified residual form for hybrid physics-informed learning and links generalization performance to affine variety dimensions, extending analysis to nonlinear systems.

Findings

Generalization governed by affine variety dimension, not parameter count

Unified residual form for collocation and variational methods

Experimental validation supports theoretical insights

Abstract

Physics-informed machine learning is gaining significant traction for enhancing statistical performance and sample efficiency through the integration of physical knowledge. However, current theoretical analyses often presume complete prior knowledge in non-hybrid settings, overlooking the crucial integration of observational data, and are frequently limited to linear systems, unlike the prevalent nonlinear nature of many real-world applications. To address these limitations, we introduce a unified residual form that unifies collocation and variational methods, enabling the incorporation of incomplete and complex physical constraints in hybrid learning settings. Within this formulation, we establish that the generalization performance of physics-informed regression in such hybrid settings is governed by the dimension of the affine variety associated with the physical constraint, rather than by the number of parameters. This enables a unified analysis that is applicable to both linear and nonlinear equations. We also present a method to approximate this dimension and provide experimental validation of our theoretical findings.