MODNO: Multi Operator Learning With Distributed Neural Operators

TL;DR

This paper introduces a distributed training method for multi-operator neural learning that reduces parameters and costs, enabling efficient approximation of multiple operators with improved accuracy and data sharing.

Contribution

It proposes a novel distributed training approach for neural operators that efficiently handles multiple operators with fewer parameters and shared input encoding.

Findings

Enhanced training efficiency and accuracy demonstrated across five numerical examples.

The method allows operators with limited data to benefit from data of similar operators.

The approach reduces model complexity without sacrificing performance.

Abstract

The study of operator learning involves the utilization of neural networks to approximate operators. Traditionally, the focus has been on single-operator learning (SOL). However, recent advances have rapidly expanded this to include the approximation of multiple operators using foundation models equipped with millions or billions of trainable parameters, leading to the research of multi-operator learning (MOL). In this paper, we present a novel distributed training approach aimed at enabling a single neural operator with significantly fewer parameters to effectively tackle multi-operator learning challenges, all without incurring additional average costs. Our method is applicable to various neural operators, such as Deep Operator Neural Networks (DON). The core idea is to independently learn the output basis functions for each operator using its dedicated data, while simultaneously centralizing the learning of the input function encoding shared by all operators using the entire dataset. Through a systematic study of five numerical examples, we compare the accuracy and cost of training a single neural operator for each operator independently versus training a MOL model using our proposed method. Our results demonstrate enhanced efficiency and satisfactory accuracy. Moreover, our approach illustrates that some operators with limited data can be more effectively constructed with the aid of data from analogous operators through MOL learning. This highlights another MOL's potential to bolster operator learning.