New affine invariant ensemble samplers and their dimensional scaling

TL;DR

This paper introduces new affine invariant ensemble MCMC samplers, including derivative-free and derivative-based Hamiltonian variants, that outperform existing methods especially in high-dimensional, anisotropic problems, with improved scaling properties.

Contribution

The paper presents novel affine invariant ensemble samplers, including a derivative-free side move and a Hamiltonian version with antisymmetric preconditioning, enhancing high-dimensional sampling efficiency.

Findings

Derivative-based affine invariant HMC scales better with dimension.

Proposed samplers outperform existing methods in high-dimensional anisotropic problems.

Asymptotic analysis elucidates the scaling advantages of the new samplers.

Abstract

We introduce new affine invariant ensemble Markov chain Monte Carlo (MCMC) samplers that are easy to construct and improve upon existing methods, especially for high-dimensional problems. We first propose a simple derivative-free side move sampler that improves upon popular samplers in the \texttt{emcee} package by generating more effective proposal directions. We then develop a class of derivative-based affine invariant ensemble Hamiltonian Monte Carlo (HMC) samplers based on antisymmetric preconditioning using complementary ensembles, which outperform standard, non-affine-invariant HMC when sampling highly anisotropic distributions. We provide asymptotic scaling analysis for high-dimensional Gaussian targets to further elucidate the properties of these affine invariant ensemble samplers. In particular, with derivative information, the affine invariant ensemble HMC can scale much better with dimension compared to derivative-free ensemble samplers.