Bounding the Rademacher Complexity of Fourier neural operators

TL;DR

This paper analyzes the generalization capabilities of Fourier neural operators by bounding their Rademacher complexity using group norms, providing insights into how model architecture affects learning performance.

Contribution

It introduces a method to bound the Rademacher complexity of FNOs based on group norms, linking model capacity to generalization error and architecture.

Findings

Bounded the generalization error of FNOs using capacity measures.

Identified the influence of group norms on model capacity and weights.

Experimental validation of the relationship between modes and generalization error.

Abstract

A Fourier neural operator (FNO) is one of the physics-inspired machine learning methods. In particular, it is a neural operator. In recent times, several types of neural operators have been developed, e.g., deep operator networks, Graph neural operator (GNO), and Multiwavelet-based operator (MWTO). Compared with other models, the FNO is computationally efficient and can learn nonlinear operators between function spaces independent of a certain finite basis. In this study, we investigated the bounding of the Rademacher complexity of the FNO based on specific group norms. Using capacity based on these norms, we bound the generalization error of the model. In addition, we investigated the correlation between the empirical generalization error and the proposed capacity of FNO. From the perspective of our result, we inferred that the type of group norms determines the information about the weights and architecture of the FNO model stored in the capacity. And then, we confirmed these inferences through experiments. Based on this fact, we gained insight into the impact of the number of modes used in the FNO model on the generalization error. And we got experimental results that followed our insights.