Diffusion-based supervised learning of generative models for efficient sampling of multimodal distributions

TL;DR

This paper introduces a hybrid diffusion-based generative modeling approach that efficiently samples high-dimensional, multimodal distributions by identifying modes, training mode-specific models, and adjusting mode proportions, outperforming traditional Monte Carlo methods.

Contribution

It presents a novel divide-and-conquer framework combining mode detection, classification, and diffusion models for multimodal sampling in high dimensions.

Findings

Effective handling of multimodal distributions up to 100 dimensions

Accurate mode proportion adjustment via bridge sampling

Successful application to Bayesian inverse problems

Abstract

We propose a hybrid generative model for efficient sampling of high-dimensional, multimodal probability distributions for Bayesian inference. Traditional Monte Carlo methods, such as the Metropolis-Hastings and Langevin Monte Carlo sampling methods, are effective for sampling from single-mode distributions in high-dimensional spaces. However, these methods struggle to produce samples with the correct proportions for each mode in multimodal distributions, especially for distributions with well separated modes. To address the challenges posed by multimodality, we adopt a divide-and-conquer strategy. We start by minimizing the energy function with initial guesses uniformly distributed within the prior domain to identify all the modes of the energy function. Then, we train a classifier to segment the domain corresponding to each mode. After the domain decomposition, we train a diffusion-model-assisted generative model for each identified mode within its support. Once each mode is characterized, we employ bridge sampling to estimate the normalizing constant, allowing us to directly adjust the ratios between the modes. Our numerical examples demonstrate that the proposed framework can effectively handle multimodal distributions with varying mode shapes in up to 100 dimensions. An application to Bayesian inverse problem for partial differential equations is also provided.