Efficient sampling from a multivariate normal distribution subject to linear equality and inequality constraints

TL;DR

This paper introduces an efficient, rejection-free method for sampling from multivariate normal distributions constrained by linear equalities and inequalities, combining elliptical slice sampling and linear mapping.

Contribution

The work presents a general framework that efficiently samples from constrained multivariate normal distributions using elliptical slice sampling and linear programming, improving over traditional rejection-based methods.

Findings

Rejection-free sampling method is more efficient than traditional methods.

Accurate estimation of mean and covariance under constraints.

Method validated on a 4D example with multiple constraints.

Abstract

Sampling from multivariate normal distributions, subjected to a variety of restrictions, is a problem that is recurrent in statistics and computing. In the present work, we demonstrate a general framework to efficiently sample a multivariate normal distribution subject to any set of linear inequality constraints and/or linear equality constraints simultaneously. In the approach we detail, sampling a multivariate random variable from the domain formed by the intersection of linear constraints proceeds via a combination of elliptical slice sampling to address the inequality constraints, and linear mapping to address the equality constraints. We also detail a linear programming method for finding an initial sample on the linearly constrained domain; such a method is critical for sampling problems where the domain has small probability. We demonstrate the validity of our methods on an arbitrarily chosen four-dimensional multivariate normal distribution subject to five inequality constraints and/or two equality constraints. Our approach compares favourably to direct sampling and/or accept-reject sampling methods; the latter methods vary widely in their efficiency, whereas the methods in the present work are rejection-free. Where practical we compare predictions of probability density functions between our sampling methods and analytical computation. For all simulations we demonstrate that our methods yield accurate computation of the mean and covariance of the multivariate normal distributions restricted by the imposed linear constraints. MATLAB codes to implement our methods are readily available at https://dx.doi.org/10.6084/m9.figshare.29956304 .