BayesSum: Bayesian Quadrature in Discrete Spaces

TL;DR

BayesSum introduces a Bayesian quadrature-based estimator for discrete expectations, leveraging prior information to achieve faster convergence and requiring fewer samples than traditional methods.

Contribution

It extends Bayesian quadrature to discrete domains, providing a more sample-efficient estimator with theoretical convergence guarantees.

Findings

Faster convergence rate than Monte Carlo in theory

Requires fewer samples in synthetic experiments

Effective for parameter estimation in complex models

Abstract

This paper addresses the challenging computational problem of estimating intractable expectations over discrete domains. Existing approaches, including Monte Carlo and Russian Roulette estimators, are consistent but often require a large number of samples to achieve accurate results. We propose a novel estimator, \emph{BayesSum}, which is an extension of Bayesian quadrature to discrete domains. It is more sample efficient than alternatives due to its ability to make use of prior information about the integrand through a Gaussian process. We show this through theory, deriving a convergence rate significantly faster than Monte Carlo in a broad range of settings. We also demonstrate empirically that our proposed method does indeed require fewer samples on several synthetic settings as well as for parameter estimation for Conway-Maxwell-Poisson and Potts models.