Bayesian Quadrature: Gaussian Processes for Integration

TL;DR

This paper provides a comprehensive survey of Bayesian quadrature, a probabilistic approach to numerical integration using Gaussian processes, covering its foundations, taxonomy, theoretical guarantees, practical challenges, and applications.

Contribution

It offers the first systematic, in-depth review of Bayesian quadrature, including taxonomy, theoretical analysis, and practical insights, filling a significant gap in the literature.

Findings

Systematic taxonomy for Bayesian quadrature methods

Theoretical guarantees for convergence and accuracy

Numerical study illustrating method choices and limitations

Abstract

Bayesian quadrature is a probabilistic, model-based approach to numerical integration, the estimation of intractable integrals, or expectations. Although Bayesian quadrature was popularised already in the 1980s, no systematic and comprehensive treatment has been published. The purpose of this survey is to fill this gap. We review the mathematical foundations of Bayesian quadrature from different points of view; present a systematic taxonomy for classifying different Bayesian quadrature methods along the three axes of modelling, inference, and sampling; collect general theoretical guarantees; and provide a controlled numerical study that explores and illustrates the effect of different choices along the axes of the taxonomy. We also provide a realistic assessment of practical challenges and limitations to application of Bayesian quadrature methods and include an up-to-date and nearly exhaustive bibliography that covers not only machine learning and statistics literature but all areas of mathematics and engineering in which Bayesian quadrature or equivalent methods have seen use.