Estimation of ratios of normalizing constants using stochastic approximation : the SARIS algorithm

TL;DR

This paper introduces SARIS, a stochastic approximation-based method for estimating ratios of normalizing constants, offering improved accuracy and robustness over existing methods, with applications in Bayesian model comparison and latent variable testing.

Contribution

The paper presents a novel stochastic approximation algorithm, SARIS, for estimating ratios of normalizing constants that outperforms existing methods in accuracy and robustness.

Findings

SARIS estimator is consistent and asymptotically Gaussian.

It has smaller asymptotic variance than optimal bridge sampling.

SARIS is more robust to little overlap between distributions.

Abstract

Computing ratios of normalizing constants plays an important role in statistical modeling. Two important examples are hypothesis testing in latent variables models, and model comparison in Bayesian statistics. In both examples, the likelihood ratio and the Bayes factor are defined as the ratio of the normalizing constants of posterior distributions. We propose in this article a novel methodology that estimates this ratio using stochastic approximation principle. Our estimator is consistent and asymptotically Gaussian. Its asymptotic variance is smaller than the one of the popular optimal bridge sampling estimator. Furthermore, it is much more robust to little overlap between the two unnormalized distributions considered. Thanks to its online definition, our procedure can be integrated in an estimation process in latent variables model, and therefore reduce the computational effort. The performances of the estimator are illustrated through a simulation study and compared to two other estimators : the ratio importance sampling and the optimal bridge sampling estimators.