Physics-informed fine-tuning of foundation models for partial differential equations

TL;DR

This paper introduces a physics-informed fine-tuning method for PDE foundation models that incorporates physical constraints into the training process, enabling effective adaptation with limited data and improved physical consistency.

Contribution

It systematically studies and demonstrates the effectiveness of physics-informed fine-tuning for PDE foundation models, a novel approach in scientific machine learning.

Findings

Achieves competitive accuracy without PDE solutions for training.

Enhances out-of-distribution generalization with hybrid fine-tuning.

Promotes physical consistency in model adaptation.

Abstract

Foundation models for partial differential equations (PDEs) have emerged as powerful surrogates pre-trained on diverse physical systems, but adapting them to new downstream tasks remains challenging due to limited task-specific data and distribution shifts. While fine-tuning has proven transformative in natural language processing, best practices for adapting PDE foundation models remain underexplored. Although physics-informed training has successfully trained accurate solvers across a wide range of PDE problems, its potential for fine-tuning data-based foundation models has not been systematically studied. In this work, we introduce a physics-informed fine-tuning framework that adapts pre-trained PDE foundation models by incorporating physical constraints (PDE residuals and boundary conditions) directly into the fine-tuning objective. This enables effective adaptation in data-scarce regimes while promoting physical consistency. We evaluate our method on a downstream task composed of an unseen PDE class and compare it with data-driven finetuning counterparts. Our results demonstrate that physics-informed fine-tuning achieves competitive accuracy without requiring PDE solutions for training. Furthermore, a hybrid fine-tuning strategy yields superior generalization to out-of-distribution scenarios when only minimal training data is available. These findings establish physics-informed fine-tuning as a scalable and data-efficient paradigm, providing a physically interpretable pathway for adapting foundation models in scientific machine learning.