Complexity Dependent Error Rates for Physics-informed Statistical Learning via the Small-ball Method

TL;DR

This paper develops a theoretical framework for physics-informed statistical learning, analyzing how physical knowledge and regularization influence error rates and model complexity in convex function classes.

Contribution

It introduces complexity-dependent error bounds using the small-ball method, comparing soft penalties and hard constraints, and demonstrates the benefits of informed penalization for reducing model complexity.

Findings

Error rates of physics-informed estimators are comparable to hard constrained methods.

Informed penalization reduces model complexity, akin to dimensionality reduction.

Provides a theoretical foundation for evaluating physics-informed estimators.

Abstract

Physics-informed statistical learning (PISL) integrates empirical data with physical knowledge to enhance the statistical performance of estimators. While PISL methods are widely used in practice, a comprehensive theoretical understanding of how informed regularization affects statistical properties is still missing. Specifically, two fundamental questions have yet to be fully addressed: (1) what is the trade-off between considering soft penalties versus hard constraints, and (2) what is the statistical gain of incorporating physical knowledge compared to purely data-driven empirical error minimisation. In this paper, we address these questions for PISL in convex classes of functions under physical knowledge expressed as linear equations by developing appropriate complexity dependent error rates based on the small-ball method. We show that, under suitable assumptions, (1) the error rates of physics-informed estimators are comparable to those of hard constrained empirical error minimisers, differing only by constant terms, and that (2) informed penalization can effectively reduce model complexity, akin to dimensionality reduction, thereby improving learning performance. This work establishes a theoretical framework for evaluating the statistical properties of physics-informed estimators in convex classes of functions, contributing to closing the gap between statistical theory and practical PISL, with potential applications to cases not yet explored in the literature.