Ensemble and Mixture-of-Experts DeepONets For Operator Learning

TL;DR

This paper introduces ensemble and mixture-of-experts DeepONet architectures that enhance expressivity and spatial locality, achieving significantly lower errors in operator learning tasks involving PDEs.

Contribution

The paper proposes novel ensemble and PoU-MoE DeepONet architectures with universal approximation properties, improving accuracy and spatial modeling in operator learning.

Findings

Ensemble DeepONets achieve 2-4x lower errors than standard models.

PoU-MoE DeepONet promotes spatial locality and sparsity.

Both architectures are proven to be universal approximators.

Abstract

We present a novel deep operator network (DeepONet) architecture for operator learning, the ensemble DeepONet, that allows for enriching the trunk network of a single DeepONet with multiple distinct trunk networks. This trunk enrichment allows for greater expressivity and generalization capabilities over a range of operator learning problems. We also present a spatial mixture-of-experts (MoE) DeepONet trunk network architecture that utilizes a partition-of-unity (PoU) approximation to promote spatial locality and model sparsity in the operator learning problem. We first prove that both the ensemble and PoU-MoE DeepONets are universal approximators. We then demonstrate that ensemble DeepONets containing a trunk ensemble of a standard trunk, the PoU-MoE trunk, and/or a proper orthogonal decomposition (POD) trunk can achieve 2-4x lower relative $\ell_2$ errors than standard DeepONets and POD-DeepONets on both standard and challenging new operator learning problems involving partial differential equations (PDEs) in two and three dimensions. Our new PoU-MoE formulation provides a natural way to incorporate spatial locality and model sparsity into any neural network architecture, while our new ensemble DeepONet provides a powerful and general framework for incorporating basis enrichment in scientific machine learning architectures for operator learning.