Transfer Learning in Physics-Informed Neural Networks: Full Fine-Tuning, Lightweight Fine-Tuning, and Low-Rank Adaptation

TL;DR

This paper investigates transfer learning techniques for Physics-Informed Neural Networks (PINNs), demonstrating that methods like full fine-tuning and Low-Rank Adaptation can accelerate convergence and slightly improve accuracy across different PDE problem settings.

Contribution

It introduces and evaluates transfer learning methods such as full fine-tuning, lightweight fine-tuning, and LoRA for PINNs to enhance their generalization and efficiency.

Findings

Full finetuning and LoRA significantly speed up convergence.

Transfer learning improves PINN accuracy with minimal retraining.

Methods adapt PINNs to various boundary conditions and geometries.

Abstract

AI for PDEs has garnered significant attention, particularly Physics-Informed Neural Networks (PINNs). However, PINNs are typically limited to solving specific problems, and any changes in problem conditions necessitate retraining. Therefore, we explore the generalization capability of transfer learning in the strong and energy form of PINNs across different boundary conditions, materials, and geometries. The transfer learning methods we employ include full finetuning, lightweight finetuning, and Low-Rank Adaptation (LoRA). The results demonstrate that full finetuning and LoRA can significantly improve convergence speed while providing a slight enhancement in accuracy.