Scalable Importance Sampling in High Dimensions with Low-Rank Mixture Proposals

TL;DR

This paper introduces a scalable importance sampling method using low-rank mixture models, specifically MPPCA, to efficiently estimate rare events in high-dimensional spaces, overcoming covariance estimation challenges.

Contribution

The paper proposes using MPPCA as a low-rank proposal distribution for importance sampling, enabling efficient high-dimensional rare event estimation.

Findings

Demonstrates improved sample efficiency in simulated systems

Shows better failure distribution characterization

Validates scalability in high-dimensional settings

Abstract

Importance sampling is a Monte Carlo technique for efficiently estimating the likelihood of rare events by biasing the sampling distribution towards the rare event of interest. By drawing weighted samples from a learned proposal distribution, importance sampling allows for more sample-efficient estimation of rare events or tails of distributions. A common choice of proposal density is a Gaussian mixture model (GMM). However, estimating full-rank GMM covariance matrices in high dimensions is a challenging task due to numerical instabilities. In this work, we propose using mixtures of probabilistic principal component analyzers (MPPCA) as the parametric proposal density for importance sampling methods. MPPCA models are a type of low-rank mixture model that can be fit quickly using expectation-maximization, even in high-dimensional spaces. We validate our method on three simulated systems, demonstrating consistent gains in sample efficiency and quality of failure distribution characterization.