Likelihood-Free Inference with Generative Neural Networks via Scoring Rule Minimization

TL;DR

This paper introduces a new adversarial-free method for training generative neural networks to perform likelihood-free Bayesian inference, resulting in more stable training and improved uncertainty quantification compared to adversarial approaches.

Contribution

It proposes using Scoring Rule minimization to train generative networks for likelihood-free inference, avoiding instability of adversarial training and enhancing uncertainty estimation.

Findings

Better performance in simulation studies

Shorter training times

More stable training process

Abstract

Bayesian Likelihood-Free Inference methods yield posterior approximations for simulator models with intractable likelihood. Recently, many works trained neural networks to approximate either the intractable likelihood or the posterior directly. Most proposals use normalizing flows, namely neural networks parametrizing invertible maps used to transform samples from an underlying base measure; the probability density of the transformed samples is then accessible and the normalizing flow can be trained via maximum likelihood on simulated parameter-observation pairs. A recent work [Ramesh et al., 2022] approximated instead the posterior with generative networks, which drop the invertibility requirement and are thus a more flexible class of distributions scaling to high-dimensional and structured data. However, generative networks only allow sampling from the parametrized distribution; for this reason, Ramesh et al. [2022] follows the common solution of adversarial training, where the generative network plays a min-max game against a "critic" network. This procedure is unstable and can lead to a learned distribution underestimating the uncertainty - in extreme cases collapsing to a single point. Here, we propose to approximate the posterior with generative networks trained by Scoring Rule minimization, an overlooked adversarial-free method enabling smooth training and better uncertainty quantification. In simulation studies, the Scoring Rule approach yields better performances with shorter training time with respect to the adversarial framework.