TransNet: Transferable Neural Networks for Partial Differential Equations

TL;DR

This paper introduces a novel transfer learning approach for PDEs that constructs neural feature spaces without PDE-specific data, achieving high transferability and accuracy across diverse PDE problems.

Contribution

It proposes a PDE-agnostic method to build transferable neural feature spaces using re-parameterization and auxiliary functions, with theoretical guarantees and superior empirical performance.

Findings

High-quality, uniformly distributed neural feature spaces.

Significantly improved transferability across different PDEs.

Orders of magnitude better accuracy than existing methods.

Abstract

Transfer learning for partial differential equations (PDEs) is to develop a pre-trained neural network that can be used to solve a wide class of PDEs. Existing transfer learning approaches require much information of the target PDEs such as its formulation and/or data of its solution for pre-training. In this work, we propose to construct transferable neural feature spaces from purely function approximation perspectives without using PDE information. The construction of the feature space involves re-parameterization of the hidden neurons and uses auxiliary functions to tune the resulting feature space. Theoretical analysis shows the high quality of the produced feature space, i.e., uniformly distributed neurons. Extensive numerical experiments verify the outstanding performance of our method, including significantly improved transferability, e.g., using the same feature space for various PDEs with different domains and boundary conditions, and the superior accuracy, e.g., several orders of magnitude smaller mean squared error than the state of the art methods.