Physics-Informed Machine Learning of Dynamical Systems for Efficient Bayesian Inference

TL;DR

This paper introduces latent variable Hamiltonian neural networks (L-HNNs) integrated into the NUTS algorithm to perform Bayesian inference more efficiently by reducing the need for numerous posterior gradients.

Contribution

The paper proposes L-HNNs for Bayesian inference, enhancing expressivity and efficiency, and integrates them into NUTS with an online error monitoring scheme.

Findings

L-HNNs improve sampling efficiency in high-dimensional posteriors.

The integrated method reduces computational cost compared to traditional NUTS.

Online error monitoring prevents sampling degeneracy.

Abstract

Although the no-u-turn sampler (NUTS) is a widely adopted method for performing Bayesian inference, it requires numerous posterior gradients which can be expensive to compute in practice. Recently, there has been a significant interest in physics-based machine learning of dynamical (or Hamiltonian) systems and Hamiltonian neural networks (HNNs) is a noteworthy architecture. But these types of architectures have not been applied to solve Bayesian inference problems efficiently. We propose the use of HNNs for performing Bayesian inference efficiently without requiring numerous posterior gradients. We introduce latent variable outputs to HNNs (L-HNNs) for improved expressivity and reduced integration errors. We integrate L-HNNs in NUTS and further propose an online error monitoring scheme to prevent sampling degeneracy in regions where L-HNNs may have little training data. We demonstrate L-HNNs in NUTS with online error monitoring considering several complex high-dimensional posterior densities and compare its performance to NUTS.