On Contrastive Learning for Likelihood-free Inference

TL;DR

This paper unifies two main classes of likelihood-free inference methods—classifier-based and density estimator-based—under a contrastive learning framework, clarifying their relationships and guiding their application.

Contribution

It introduces a unified contrastive learning perspective for likelihood-free inference, providing theoretical insights and practical guidance for using these methods.

Findings

Both approaches can be viewed as contrastive learning methods.

The paper clarifies how to properly run and compare these methods.

Provides a theoretical framework connecting different likelihood-free inference techniques.

Abstract

Likelihood-free methods perform parameter inference in stochastic simulator models where evaluating the likelihood is intractable but sampling synthetic data is possible. One class of methods for this likelihood-free problem uses a classifier to distinguish between pairs of parameter-observation samples generated using the simulator and pairs sampled from some reference distribution, which implicitly learns a density ratio proportional to the likelihood. Another popular class of methods fits a conditional distribution to the parameter posterior directly, and a particular recent variant allows for the use of flexible neural density estimators for this task. In this work, we show that both of these approaches can be unified under a general contrastive learning scheme, and clarify how they should be run and compared.