Unbiased approximations of products of expectations

TL;DR

This paper introduces an efficient unbiased estimator for the product of multiple expectations, reducing the computational complexity from quadratic to linear in the number of expectations, with applications in Bayesian computation and gene expression analysis.

Contribution

The paper proposes a novel unbiased estimator that approximates each expectation efficiently, significantly reducing the number of particles needed compared to standard methods.

Findings

Estimator requires only O(n) particles, matching quadratic methods with fewer resources.

Demonstrated computational gains in Bayesian and gene expression applications.

Achieves unbiasedness while improving efficiency in latent variable models.

Abstract

We consider the problem of approximating the product of $n$ expectations with respect to a common probability distribution $\mu$. Such products routinely arise in statistics as values of the likelihood in latent variable models. Motivated by pseudo-marginal Markov chain Monte Carlo schemes, we focus on unbiased estimators of such products. The standard approach is to sample $N$ particles from $\mu$ and assign each particle to one of the expectations. This is wasteful and typically requires the number of particles to grow quadratically with the number of expectations. We propose an alternative estimator that approximates each expectation using most of the particles while preserving unbiasedness. We carefully study its properties, showing that in latent variable contexts the proposed estimator needs only $\mathcal{O}(n)$ particles to match the performance of the standard approach with $\mathcal{O}(n^{2})$ particles. We demonstrate the procedure on two latent variable examples from approximate Bayesian computation and single-cell gene expression analysis, observing computational gains of the order of the number of expectations, i.e. data points, $n$.