Fine-Tuning DeepONets to Enhance Physics-informed Neural Networks for solving Partial Differential Equations

TL;DR

This paper introduces a parameter-efficient fine-tuning method for DeepONet models within PINNs, significantly reducing training time for PDE solutions while maintaining accuracy and improving generalization.

Contribution

It proposes a novel FTO-PINN approach that fine-tunes pre-trained DeepONets with minimal parameters, enhancing efficiency and performance in solving PDEs.

Findings

FTO-PINN reduces training time compared to vanilla PINNs.

The method maintains accuracy comparable to traditional PINNs.

FTO-PINN outperforms pre-trained DeepONet in fidelity and generalization.

Abstract

Physics-Informed Neural Networks (PINNs) have emerged as powerful tools for solving partial differential equations (PDEs). However, training PINNs from scratch is often computationally intensive and time-consuming. To address this problem, we propose a parameter-efficient approach that fine-tunes pre-trained DeepONet models within the PINN framework (FTO-PINN), enabling more efficient meshless PDE solving. Specifically, we freeze the weights of the pre-trained DeepONet model and fine-tune the output of the branch net by incorporating a small number of new trainable parameters, which can be quickly determined using least-squares techniques. Additionally, we introduce trunk net expansions and low-rank adaptation strategies to further enhance the performance of FTO-PINN. The effectiveness of our proposed method is demonstrated through a series of numerical experiments across various types of PDEs. FTO-PINN significantly reduces the training time of vanilla PINNs while maintaining comparable accuracy, and outperforms DeepONet, which is pre-trained on general function data, in both fidelity and generalization capabilities.