A Review of Diffusion-based Simulation-Based Inference: Foundations and Applications in Non-Ideal Data Scenarios

TL;DR

This review explores diffusion-based simulation-based inference methods, emphasizing their mathematical foundations, applications in non-ideal data scenarios, and addressing challenges like model mismatch and missing data.

Contribution

It synthesizes recent diffusion SBI methods, highlighting their robustness and applicability to complex scientific problems with non-ideal data conditions.

Findings

Diffusion models improve robustness in likelihood-free inference.

Survey of eight methods addressing non-ideal data scenarios.

Discussion of open problems and applications in geophysical uncertainty.

Abstract

For complex simulation problems, inferring parameters often precludes the use of classical likelihood-based techniques due to intractable likelihoods. Simulation-based inference (SBI) methods offer a likelihood-free approach to directly learn posterior distributions $p(\bftheta \mid \xobs)$ from simulator outputs. Recently, diffusion models have emerged as promising tools for SBI, addressing limitations of earlier neural methods such as neural likelihood/posterior estimation and normalizing flows. This review examines diffusion-based SBI from first principles to applications, emphasizing robustness in three non-ideal data scenarios common to scientific computing: model misspecification (simulator-reality mismatch), unstructured or infinite-dimensional observations, and missing data. We synthesize mathematical foundations and survey eight methods addressing these challenges, such as conditional diffusion for irregular data, guided diffusion for prior adaptation, sequential and factorized approaches for efficiency, and consistency models for fast sampling. Throughout, we maintain consistent notation and emphasize conditions required for accurate posteriors. We conclude with open problems and applications to geophysical uncertainty quantification, where these challenges are acute.