Projected Wasserstein gradient descent for high-dimensional Bayesian inference

TL;DR

This paper introduces a projected Wasserstein gradient descent method (pWGD) that efficiently performs high-dimensional Bayesian inference by leveraging low-rank structures and projecting parameters into low-dimensional subspaces, improving scalability and accuracy.

Contribution

The paper develops a novel pWGD algorithm that overcomes the curse of dimensionality in KDE-based Bayesian inference by exploiting low-rank structures and parameter projection.

Findings

pWGD demonstrates high accuracy in high-dimensional settings.

The method shows favorable convergence properties.

Scalability improves with increased parameter dimensions and computational resources.

Abstract

We propose a projected Wasserstein gradient descent method (pWGD) for high-dimensional Bayesian inference problems. The underlying density function of a particle system of WGD is approximated by kernel density estimation (KDE), which faces the long-standing curse of dimensionality. We overcome this challenge by exploiting the intrinsic low-rank structure in the difference between the posterior and prior distributions. The parameters are projected into a low-dimensional subspace to alleviate the approximation error of KDE in high dimensions. We formulate a projected Wasserstein gradient flow and analyze its convergence property under mild assumptions. Several numerical experiments illustrate the accuracy, convergence, and complexity scalability of pWGD with respect to parameter dimension, sample size, and processor cores.