Towards Physics-Guided Foundation Models

TL;DR

This paper introduces physics-guided foundation models (PGFM), integrating broad scientific physical knowledge into foundation models to improve out-of-distribution prediction and physical realism in outputs.

Contribution

It proposes a novel framework for incorporating physical knowledge into foundation models, enhancing their applicability in scientific domains.

Findings

Improved out-of-distribution prediction accuracy

Enhanced physical plausibility of model outputs

Framework applicable across various scientific tasks

Abstract

Traditional foundation models are pre-trained on broad datasets to reduce the training resources (e.g., time, energy, labeled samples) needed for fine-tuning a wide range of downstream tasks. However, traditional foundation models struggle with out-of-distribution prediction and can produce outputs that are unrealistic and physically infeasible. We propose the notation of physics-guided foundation models (PGFM), that is, foundation models integrated with broad or general domain (e.g., scientific) physical knowledge applicable to a wide range of downstream tasks.