Unreasonable effectiveness of Monte Carlo

TL;DR

This paper comments on the surprising effectiveness of classical numerical integration methods, highlighting the challenges and limitations faced by probabilistic numerics despite its potential advantages.

Contribution

It critically examines the reasons behind the strong performance of classical methods and discusses the challenges faced by probabilistic approaches in numerical integration.

Findings

Classical methods outperform probabilistic ones in certain integration tasks.

The central limit theorem and pseudorandom generators contribute to classical methods' effectiveness.

Probabilistic methods face computational challenges like $O(n^3)$ costs.

Abstract

This is a comment on the article "Probabilistic Integration: A Role in Statistical Computation?" by F.-X. Briol, C. J. Oates, M. Girolami, M. A. Osborne and D. Sejdinovic to appear in Statistical Science. There is a role for statistical computation in numerical integration. However, the competition from incumbent methods looks to be stiffer for this problem than for some of the newer problems being handled by probabilistic numerics. One of the challenges is the unreasonable effectiveness of the central limit theorem. Another is the unreasonable effectiveness of pseudorandom number generators. A third is the common $O(n^3)$ cost of methods based on Gaussian processes. Despite these advantages, the classical methods are weak in places where probabilistic methods could bring an improvement.