Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference

TL;DR

This paper presents two novel synthetic likelihood methods for simulation-based inference that leverage energy-based models and KL minimization, enabling efficient posterior estimation from high-fidelity simulators.

Contribution

The authors introduce two synthetic likelihood approaches using energy-based models and KL loss, offering an alternative to existing methods reliant on normalizing flows or score-based objectives.

Findings

Methods outperform prior art on synthetic datasets.

Approach reduces simulation budget in neuroscience model.

Flexible energy-based likelihood modeling enhances inference accuracy.

Abstract

We introduce two synthetic likelihood methods for Simulation-Based Inference (SBI), to conduct either amortized or targeted inference from experimental observations when a high-fidelity simulator is available. Both methods learn a conditional energy-based model (EBM) of the likelihood using synthetic data generated by the simulator, conditioned on parameters drawn from a proposal distribution. The learned likelihood can then be combined with any prior to obtain a posterior estimate, from which samples can be drawn using MCMC. Our methods uniquely combine a flexible Energy-Based Model and the minimization of a KL loss: this is in contrast to other synthetic likelihood methods, which either rely on normalizing flows, or minimize score-based objectives; choices that come with known pitfalls. We demonstrate the properties of both methods on a range of synthetic datasets, and apply them to a neuroscience model of the pyloric network in the crab, where our method outperforms prior art for a fraction of the simulation budget.