DeepONet as a Multi-Operator Extrapolation Model: Distributed Pretraining with Physics-Informed Fine-Tuning

TL;DR

This paper introduces a novel transfer learning framework combining distributed pretraining and physics-informed fine-tuning to improve multi-operator learning for PDE-related problems, enabling rapid adaptation with minimal data.

Contribution

It presents a new method for multi-operator neural operator learning that leverages distributed pretraining and physics-informed zero-shot fine-tuning, enhancing generalization to new tasks.

Findings

Significant accuracy improvements demonstrated in numerical examples

Effective zero-shot fine-tuning with physics-informed losses

Enhanced generalization to complex nonlinear operators

Abstract

We propose a novel fine-tuning method to achieve multi-operator learning through training a distributed neural operator with diverse function data and then zero-shot fine-tuning the neural network using physics-informed losses for downstream tasks. Operator learning effectively approximates solution operators for PDEs and various PDE-related problems, yet it often struggles to generalize to new tasks. To address this, we investigate fine-tuning a pretrained model, while carefully selecting an initialization that enables rapid adaptation to new tasks with minimal data. Our approach combines distributed learning to integrate data from various operators in pre-training, while physics-informed methods enable zero-shot fine-tuning, minimizing the reliance on downstream data. We investigate standard fine-tuning and Low-Rank Adaptation fine-tuning, applying both to train complex nonlinear target operators that are difficult to learn only using random initialization. Through comprehensive numerical examples, we demonstrate the advantages of our approach, showcasing significant improvements in accuracy. Our findings provide a robust framework for advancing multi-operator learning and highlight the potential of transfer learning techniques in this domain.