Some Best Practices in Operator Learning

TL;DR

This paper investigates hyperparameter choices and training methods for operator learning architectures like DeepONets, Fourier neural operators, and Koopman autoencoders, aiming to identify robust training trends across differential equations.

Contribution

It provides practical guidelines on hyperparameters and training strategies tailored for operator learning models, enhancing their robustness and efficiency.

Findings

Activation functions significantly affect performance.

Dropout and stochastic weight averaging improve robustness.

Certain hyperparameter settings are consistently effective across models.

Abstract

Hyperparameters searches are computationally expensive. This paper studies some general choices of hyperparameters and training methods specifically for operator learning. It considers the architectures DeepONets, Fourier neural operators and Koopman autoencoders for several differential equations to find robust trends. Some options considered are activation functions, dropout and stochastic weight averaging.