Hit and Run ARMS: Adaptive Rejection Metropolis Sampling with Hit and Run Random Direction

TL;DR

This paper introduces HARARMS, an improved sampling algorithm combining hit and run techniques with ARMS, effectively exploring complex multivariate distributions and overcoming limitations of traditional methods in high-dimensional spaces.

Contribution

The paper proposes HARARMS, a novel sampling method that updates samples in arbitrary directions, reducing trapping in subspaces and improving performance over ARMS in multidimensional distributions.

Findings

HARARMS outperforms ARMS in high-dimensional sampling tasks.

It effectively avoids trapping in isolated subspaces.

Demonstrated success in Bayesian spline regression for global knot optimization.

Abstract

An algorithm for sampling from non-log-concave multivariate distributions is proposed, which improves the adaptive rejection Metropolis sampling (ARMS) algorithm by incorporating the hit and run sampling. It is not rare that the ARMS is trapped away from some subspace with significant probability in the support of the multivariate distribution. While the ARMS updates samples only in the directions that are parallel to dimensions, our proposed method, the hit and run ARMS (HARARMS), updates samples in arbitrary directions determined by the hit and run algorithm, which makes it almost not possible to be trapped in any isolated subspaces. The HARARMS performs the same as ARMS in a single dimension while more reliable in multidimensional spaces. Its performance is illustrated by a Bayesian free-knot spline regression example. We showed that it overcomes the well-known `lethargy' property and decisively find the global optimal number and locations of the knots of the spline function.