Metropolis-Hastings Sampling Using Multivariate Gaussian Tangents

TL;DR

This paper introduces MH-MGT, a multivariate Metropolis-Hastings sampling method using Gaussian proposals based on Taylor expansion, which improves efficiency for sampling from complex, log-concave distributions, especially in high-dimensional Bayesian models.

Contribution

The paper presents MH-MGT, a novel multivariate sampling technique that leverages Taylor-series expansions for Gaussian proposals, enhancing sampling efficiency for complex log-concave densities.

Findings

MH-MGT achieves 6x efficiency improvement over univariate slice sampling.

The method is well-suited for computationally expensive log-densities with many observations.

Log-concavity is invariant under linear transformations, broadening applicability.

Abstract

We present MH-MGT, a multivariate technique for sampling from twice-differentiable, log-concave probability density functions. MH-MGT is Metropolis-Hastings sampling using asymmetric, multivariate Gaussian proposal functions constructed from Taylor-series expansion of the log-density function. The mean of the Gaussian proposal function represents the full Newton step, and thus MH-MGT is the stochastic counterpart to Newton optimization. Convergence analysis shows that MH-MGT is well suited for sampling from computationally-expensive log-densities with contributions from many independent observations. We apply the technique to Gibbs sampling analysis of a Hierarchical Bayesian marketing effectiveness model built for a large US foodservice distributor. Compared to univariate slice sampling, MH-MGT shows 6x improvement in sampling efficiency, measured in terms of `function evaluation equivalents per independent sample'. To facilitate wide applicability of MH-MGT to statistical models, we prove that log-concavity of a twice-differentiable distribution is invariant with respect to 'linear-projection' transformations including, but not restricted to, generalized linear models.