Deterministic Sampling of Expensive Posteriors Using Minimum Energy Designs

TL;DR

This paper introduces an efficient algorithm for deterministic sampling of expensive posterior distributions using Minimum Energy Designs, improving high-dimensional performance and reducing the number of costly evaluations compared to traditional methods.

Contribution

The authors develop a computationally efficient MED algorithm with enhancements for high-dimensional spaces, addressing limitations of previous implementations.

Findings

MED outperforms MCMC and QMC in example applications

The new algorithm requires fewer posterior evaluations

Improvements enhance high-dimensional sampling quality

Abstract

Markov chain Monte Carlo (MCMC) methods require a large number of samples to approximate a posterior distribution, which can be costly when the likelihood or prior is expensive to evaluate. The number of samples can be reduced if we can avoid repeated samples and those that are close to each other. This is the idea behind deterministic sampling methods such as Quasi-Monte Carlo (QMC). However, the existing QMC methods aim at sampling from a uniform hypercube, which can miss the high probability regions of the posterior distribution and thus the approximation can be poor. Minimum energy design (MED) is a recently proposed deterministic sampling method, which makes use of the posterior evaluations to obtain a weighted space-filling design in the region of interest. However, the existing implementation of MED is inefficient because it requires several global optimizations and thus numerous evaluations of the posterior. In this article, we develop an efficient algorithm that can generate MED with few posterior evaluations. We also make several improvements to the MED criterion to make it perform better in high dimensions. The advantages of MED over MCMC and QMC are illustrated using an example of calibrating a friction drilling process.