Leveraging viscous Hamilton-Jacobi PDEs for uncertainty quantification in scientific machine learning

TL;DR

This paper introduces a novel connection between Bayesian inference in scientific machine learning and viscous Hamilton-Jacobi PDEs, enabling efficient, real-time uncertainty quantification and model updates without retraining.

Contribution

It establishes a new theoretical link between Bayesian inference and viscous HJ PDEs, and develops a Riccati-based method for efficient, continuous model updating in SciML.

Findings

Riccati-based approach improves computational efficiency for model updates.

Method allows data streaming and real-time uncertainty quantification.

Demonstrated advantages in noisy data and epistemic uncertainty scenarios.

Abstract

Uncertainty quantification (UQ) in scientific machine learning (SciML) combines the powerful predictive power of SciML with methods for quantifying the reliability of the learned models. However, two major challenges remain: limited interpretability and expensive training procedures. We provide a new interpretation for UQ problems by establishing a new theoretical connection between some Bayesian inference problems arising in SciML and viscous Hamilton-Jacobi partial differential equations (HJ PDEs). Namely, we show that the posterior mean and covariance can be recovered from the spatial gradient and Hessian of the solution to a viscous HJ PDE. As a first exploration of this connection, we specialize to Bayesian inference problems with linear models, Gaussian likelihoods, and Gaussian priors. In this case, the associated viscous HJ PDEs can be solved using Riccati ODEs, and we develop a new Riccati-based methodology that provides computational advantages when continuously updating the model predictions. Specifically, our Riccati-based approach can efficiently add or remove data points to the training set invariant to the order of the data and continuously tune hyperparameters. Moreover, neither update requires retraining on or access to previously incorporated data. We provide several examples from SciML involving noisy data and \textit{epistemic uncertainty} to illustrate the potential advantages of our approach. In particular, this approach's amenability to data streaming applications demonstrates its potential for real-time inferences, which, in turn, allows for applications in which the predicted uncertainty is used to dynamically alter the learning process.