Sequential Exchange Monte Carlo: A Sampling Method for Bayesian Data Analysis without Parameter Tuning

TL;DR

This paper introduces Sequential Exchange Monte Carlo (SEMC), a new sampling method for Bayesian data analysis that requires minimal parameter tuning and effectively handles complex, multimodal distributions in materials science applications.

Contribution

The paper proposes SEMC, a novel sampling algorithm that reduces the need for parameter tuning compared to existing methods like replica exchange Monte Carlo.

Findings

SEMC achieves robust convergence without additional tuning.

SEMC outperforms NRPT and SMCS in complex multimodal distributions.

SEMC is practical for Bayesian inference in materials science.

Abstract

Bayesian data analysis is widely used across many disciplines, and representative examples in materials science include spectral analysis and sparse modeling. In such applications, the underlying models often become complex and yield multimodal posterior distributions, making efficient sampling from multimodal distributions essential. Replica exchange Monte Carlo has been commonly employed for this purpose; however, its performance strongly depends on difficult parameter tuning, such as the design of the inverse temperature. In this study, we comparatively investigate sampling algorithms that require fewer tuning parameters for Bayesian data analysis in materials science. Specifically, we compare three approaches: non-reversible parallel tempering (NRPT), sequential Monte Carlo samplers (SMCS), and a newly proposed method, sequential exchange Monte Carlo (SEMC). Our results indicate that NRPT can require computational time for parameter tuning, while SMCS requires careful adjustment of the number of MCMC steps at each temperature level. In contrast, SEMC achieves robust convergence across a range of problem settings without additional tuning, demonstrating its practicality for Bayesian inference.