Towards Universal Neural Operators through Multiphysics Pretraining

TL;DR

This paper explores the use of transformer-based neural operators pretrained on simple problems to efficiently solve complex PDEs, demonstrating effective transfer learning across diverse physical simulation tasks.

Contribution

It introduces a general transfer learning framework for transformer-based neural operators applied to multiple PDE problems, extending their applicability beyond specific cases.

Findings

Transformers effectively transfer knowledge across diverse PDE tasks.

Pretraining reduces computational costs for complex physical simulations.

Neural operators generalize well to unseen parameters and new variables.

Abstract

Although neural operators are widely used in data-driven physical simulations, their training remains computationally expensive. Recent advances address this issue via downstream learning, where a model pretrained on simpler problems is fine-tuned on more complex ones. In this research, we investigate transformer-based neural operators, which have previously been applied only to specific problems, in a more general transfer learning setting. We evaluate their performance across diverse PDE problems, including extrapolation to unseen parameters, incorporation of new variables, and transfer from multi-equation datasets. Our results demonstrate that advanced neural operator architectures can effectively transfer knowledge across PDE problems.