The Statistical Accuracy of Neural Posterior and Likelihood Estimation

TL;DR

This paper investigates the statistical properties of neural posterior and likelihood estimation methods, demonstrating they have similar guarantees to traditional approaches but often with reduced computational costs, supported by theoretical and empirical evidence.

Contribution

It provides the first in-depth theoretical analysis of NPE and NLE, showing their guarantees match ABC and BSL, with potential for more efficient inference.

Findings

NPE and NLE have similar statistical guarantees as ABC and BSL.

These methods can achieve comparable accuracy with less computational effort.

Theoretical and empirical validation of the methods' effectiveness.

Abstract

Neural posterior estimation (NPE) and neural likelihood estimation (NLE) are machine learning approaches that provide accurate posterior, and likelihood, approximations in complex modeling scenarios, and in situations where conducting amortized inference is a necessity. While such methods have shown significant promise across a range of diverse scientific applications, the statistical accuracy of these methods is so far unexplored. In this manuscript, we give, for the first time, an in-depth exploration on the statistical behavior of NPE and NLE. We prove that these methods have similar theoretical guarantees to common statistical methods like approximate Bayesian computation (ABC) and Bayesian synthetic likelihood (BSL). While NPE and NLE methods are just as accurate as ABC and BSL, we prove that this accuracy can often be achieved at a vastly reduced computational cost, and will therefore deliver more attractive approximations than ABC and BSL in certain problems. We verify our results theoretically and in several examples from the literature.