Computing Bayes: From Then 'Til Now'

TL;DR

This paper reviews the historical development of Bayesian computation from the 18th century to modern methods, highlighting key contributions and recent advances like MCMC, Hamiltonian Monte Carlo, and approximate techniques.

Contribution

It provides a comprehensive historical overview and synthesis of Bayesian computational methods, emphasizing the evolution from foundational work to current state-of-the-art algorithms.

Findings

Historical milestones in Bayesian computation highlighted

Modern methods like MCMC and Hamiltonian Monte Carlo explained

Recent approximate methods summarized

Abstract

This paper takes the reader on a journey through the history of Bayesian computation, from the 18th century to the present day. Beginning with the one-dimensional integral first confronted by Bayes in 1763, we highlight the key contributions of: Laplace, Metropolis (and, importantly, his co-authors!), Hammersley and Handscomb, and Hastings, all of which set the foundations for the computational revolution in the late 20th century -- led, primarily, by Markov chain Monte Carlo (MCMC) algorithms. A very short outline of 21st century computational methods -- including pseudo-marginal MCMC, Hamiltonian Monte Carlo, sequential Monte Carlo, and the various `approximate' methods -- completes the paper.