An Easy to Interpret Diagnostic for Approximate Inference: Symmetric Divergence Over Simulations

TL;DR

This paper proposes a new diagnostic method for approximate inference algorithms that estimates symmetric KL-divergence by simulating datasets from the prior and performing inference, addressing limitations of existing diagnostics.

Contribution

It introduces a novel diagnostic based on symmetric divergence over simulations, suitable for approximate inference methods like variational inference.

Findings

The diagnostic effectively estimates inference errors.

It works for various approximate inference algorithms.

Provides a practical tool for assessing inference quality.

Abstract

It is important to estimate the errors of probabilistic inference algorithms. Existing diagnostics for Markov chain Monte Carlo methods assume inference is asymptotically exact, and are not appropriate for approximate methods like variational inference or Laplace's method. This paper introduces a diagnostic based on repeatedly simulating datasets from the prior and performing inference on each. The central observation is that it is possible to estimate a symmetric KL-divergence defined over these simulations.