Generalization Bounds and Statistical Guarantees for Multi-Task and Multiple Operator Learning with MNO Networks

TL;DR

This paper develops a theoretical framework providing explicit generalization bounds and sample complexity guarantees for multi-task operator learning using MNO networks, based on covering numbers and approximation theory.

Contribution

It introduces a covering-number-based analysis for MNO architectures, deriving explicit bounds and tradeoffs for generalization error in hierarchical operator learning.

Findings

Derived explicit metric-entropy bounds for hypothesis classes.

Established an approximation-estimation tradeoff for test error.

Provided a sample complexity characterization for operator generalization.

Abstract

Multiple operator learning concerns learning operator families $\{G[\alpha]:U\to V\}_{\alpha\in W}$ indexed by an operator descriptor $\alpha$. Training data are collected hierarchically by sampling operator instances $\alpha$, then input functions $u$ per instance, and finally evaluation points $x$ per input, yielding noisy observations of $G[\alpha][u](x)$. While recent work has developed expressive multi-task and multiple operator learning architectures and approximation-theoretic scaling laws, quantitative statistical generalization guarantees remain limited. We provide a covering-number-based generalization analysis for separable models, focusing on the Multiple Neural Operator (MNO) architecture: we first derive explicit metric-entropy bounds for hypothesis classes given by linear combinations of products of deep ReLU subnetworks, and then combine these complexity bounds with approximation guarantees for MNO to obtain an explicit approximation-estimation tradeoff for the expected test error on new (unseen) triples $(\alpha,u,x)$. The resulting bound makes the dependence on the hierarchical sampling budgets $(n_\alpha,n_u,n_x)$ transparent and yields an explicit learning-rate statement in the operator-sampling budget $n_\alpha$, providing a sample-complexity characterization for generalization across operator instances. The structure and architecture can also be viewed as a general purpose solver or an example of a "small'' PDE foundation model, where the triples are one form of multi-modality.