Monte Carlo convergence of rival samplers

TL;DR

This paper introduces a new Monte Carlo divergence error criterion based on Jensen-Shannon divergence, enabling adaptive sampling strategies for multiple distributions under limited computational resources.

Contribution

It proposes a novel divergence-based error criterion and provides approximations for adaptive sample size allocation in parallel sampling tasks.

Findings

New Jensen-Shannon divergence-based error criterion introduced

Derived approximations enable adaptive sampling during execution

Supports optimal resource allocation among multiple samplers

Abstract

It is often necessary to make sampling-based statistical inference about many probability distributions in parallel. Given a finite computational resource, this article addresses how to optimally divide sampling effort between the samplers of the different distributions. Formally approaching this decision problem requires both the specification of an error criterion to assess how well each group of samples represent their underlying distribution, and a loss function to combine the errors into an overall performance score. For the first part, a new Monte Carlo divergence error criterion based on Jensen-Shannon divergence is proposed. Using results from information theory, approximations are derived for estimating this criterion for each target based on a single run, enabling adaptive sample size choices to be made during sampling.