Theoretical guidelines for annealed Langevin dynamics in compositional simulation-based inference

TL;DR

This paper provides theoretical guidelines for tuning annealed Langevin dynamics in compositional simulation-based inference, ensuring controlled bias and sampling accuracy through explicit decision rules derived from Wasserstein bounds.

Contribution

It derives Wasserstein bounds for annealed Langevin with approximate scores and offers explicit hyperparameter tuning rules for improved sampling in SBI.

Findings

Linhart et al.'s densities allow larger step sizes and fewer Langevin steps.

The Gaussian analysis provides closed-form expressions for hyperparameter bounds.

Empirical results show Gaussian-based tuning generalizes to complex problems.

Abstract

Compositional score-based approaches to simulation-based inference (SBI) approximate the posterior over a shared parameter given $n$ independent observations by aggregating individually learned posterior scores: currently, there are two main propositions of such methods (Geffner et al. (2023), Linhart et al. (2026)). As the resulting composite score does not correspond to the score of any distribution along the forward diffusion path of the true multi-observation posterior, sampling from it via a reverse SDE leads to an irreducible bias. Annealed Langevin dynamics provides a principled alternative: it treats the composite score as the genuine score of a sequence of tractable bridging densities and samples from them in succession. When properly tuned, it could lead to a controllable bias. However, its hyperparameters, namely step sizes, the number of steps per level, and the number of annealing levels, have so far been chosen empirically. We derive Wasserstein bounds for annealed Langevin with approximate scores and translate them into explicit decision rules for these hyperparameters that guarantee a prescribed sampling accuracy, while highlighting different theoretical aspects of each composite score formulation. In the Gaussian setting, we obtain closed-form expressions for all relevant quantities and prove that the bridging densities of Linhart et al. (2026) consistently admit larger step sizes and require fewer total Langevin steps than those of Geffner et al. (2023). Furthermore, we show empirically that the tuning obtained in the Gaussian setting generalizes to more complex problems, thus providing a well-understood and theoretically grounded starting point for practitioners using compositional score-based approaches.