Latent Neural Operator Pretraining for Solving Time-Dependent PDEs

TL;DR

This paper introduces Latent Neural Operator Pretraining (LNOP), a framework that pretrains neural operators on hybrid PDE datasets to improve solving time-dependent PDEs with higher accuracy, transferability, and data efficiency.

Contribution

The paper proposes a novel LNOP framework that pretrains neural operators on hybrid datasets, enabling effective transfer learning for various time-dependent PDEs in the latent space.

Findings

Reduces solution error by 31.7% on four problems

Achieves 57.1% error reduction after finetuning

Outperforms non-pretrained neural operators in accuracy and data efficiency

Abstract

Pretraining methods gain increasing attraction recently for solving PDEs with neural operators. It alleviates the data scarcity problem encountered by neural operator learning when solving single PDE via training on large-scale datasets consisting of various PDEs and utilizing shared patterns among different PDEs to improve the solution precision. In this work, we propose the Latent Neural Operator Pretraining (LNOP) framework based on the Latent Neural Operator (LNO) backbone. We achieve universal transformation through pretraining on hybrid time-dependent PDE dataset to extract representations of different physical systems and solve various time-dependent PDEs in the latent space through finetuning on single PDE dataset. Our proposed LNOP framework reduces the solution error by 31.7% on four problems and can be further improved to 57.1% after finetuning. On out-of-distribution dataset, our LNOP model achieves roughly 50% lower error and 3$\times$ data efficiency on average across different dataset sizes. These results show that our method is more competitive in terms of solution precision, transfer capability and data efficiency compared to non-pretrained neural operators.