Bayesian polynomial neural networks and polynomial neural ordinary differential equations

TL;DR

This paper introduces Bayesian inference methods for polynomial neural networks and polynomial neural ODEs, improving their robustness to noisy data and enabling uncertainty quantification in scientific modeling.

Contribution

It develops and validates Bayesian inference techniques, especially the Laplace approximation, for polynomial neural networks and neural ODEs, addressing noise and uncertainty.

Findings

Laplace approximation outperforms other Bayesian methods for these models

Bayesian methods enable uncertainty quantification in symbolic regression

The approach can be extended to broader symbolic neural networks

Abstract

Symbolic regression with polynomial neural networks and polynomial neural ordinary differential equations (ODEs) are two recent and powerful approaches for equation recovery of many science and engineering problems. However, these methods provide point estimates for the model parameters and are currently unable to accommodate noisy data. We address this challenge by developing and validating the following Bayesian inference methods: the Laplace approximation, Markov Chain Monte Carlo (MCMC) sampling methods, and variational inference. We have found the Laplace approximation to be the best method for this class of problems. Our work can be easily extended to the broader class of symbolic neural networks to which the polynomial neural network belongs.