A comparison of Markov Chain Monte Carlo algorithms for Bayesian inference of constitutive models

TL;DR

This paper compares three MCMC algorithms for Bayesian calibration of constitutive models, evaluating their convergence and efficiency across two physical systems using KL divergence and heuristic diagnostics.

Contribution

It provides a systematic comparison of Metropolis-Hastings, AISM, and NUTS samplers in different physical contexts, highlighting their relative performance and computational efficiency.

Findings

NUTS performs well for viscous flow due to high effective sample size.

Heuristic indicators correlate well with KL-divergence in both systems.

Gradient-based NUTS is more efficient when model evaluations are inexpensive.

Abstract

Employing Bayesian inference to calibrate constitutive model parameters has grown substantially in recent years. Among the available techniques, Markov Chain Monte Carlo (MCMC) sampling remains one of the most widely used approaches for estimating the posterior distribution. Nevertheless, the selection of a specific MCMC algorithm is often driven by practical considerations, such as software availability or prior user experience. To support sampler selection, we present a comparison of three prominent samplers in the context of two distinct physical systems: a thermal conduction system and a viscous flow system. Calibration data are obtained through tailor-made experimental setups. We use the Kullback-Leibler (KL) divergence, which quantifies the statistical distance between the sampled posterior and the reference ('true') posterior, as a measure of convergence to compare the performance of the following MCMC sampling methods: the Metropolis-Hastings (MH) sampler, the Affine Invariant Stretch Move (AISM) sampler, and the No-U-Turn Sampler (NUTS). We study how this metric correlates to heuristic indicators such as the Gelman-Rubin diagnostic and the effective sample size. In addition, we assess the samplers' computational effort in terms of required number of model evaluations. Based on the results, we find that the heuristic convergence and performance indicators provide a good qualitative measure for KL-divergence for both systems. Regarding computational effort, the NUTS is net beneficial for the viscous flow system, as the high effective sample size outweighs the additional effort required for gradient-based proposal generation. For the thermal conduction system, which involves more expensive model evaluations, the NUTS is not advantageous. Thus, the computational efficiency of gradient evaluations is an important argument in sampler selection.