Physics informed learning of orthogonal features with applications in solving partial differential equations

TL;DR

This paper introduces a physics-informed pretraining framework for orthogonal features that significantly enhances the accuracy and transferability of PDE solutions compared to standard methods.

Contribution

The paper proposes the PD-OFM framework, combining physics-informed pretraining and orthogonality regularization to improve PDE approximation and generalization.

Findings

Achieves 2-3 orders of magnitude lower residual errors in PDE solutions.

Pretrained features generalize well across different source terms and geometries.

Theoretical and empirical evidence supports improved approximation capabilities.

Abstract

The random feature method (RFM) constructs approximation spaces by initializing features from generic distributions, which provides universal approximation properties to solve general partial differential equations. However, such standard initializations lack awareness of the underlying physical laws and geometry, which limits approximation. In this work, we propose the Physics-Driven Orthogonal Feature Method (PD-OFM), a framework for constructing feature representations that are explicitly tailored to both the differential operator and the computational domain by pretraining features using physics-informed objectives together with orthogonality regularization. This pretraining strategy yields nearly orthogonal feature bases. We provide both theoretical and empirical evidence that physics-informed pretraining improves the approximation capability of the learned feature space. When employed to solve Helmholtz, Poisson, wave, and Navier-Stokes equations, the proposed method achieves residual errors 2-3 orders of magnitude lower than those of comparable methods. Furthermore, the orthogonality regularization improves transferability, enabling pretrained features to generalize effectively across different source terms and domain geometries for the same PDE.