Understanding the Hastings Algorithm

TL;DR

This paper provides intuitive derivations of the Hastings algorithm, clarifying its conceptual basis and demonstrating that the Metropolis-Hastings algorithm maximizes acceptance probability among Hastings algorithms.

Contribution

It introduces two intuitive methods to understand the Hastings algorithm, enhancing conceptual clarity and highlighting the optimality of Metropolis-Hastings.

Findings

Metropolis-Hastings has the highest acceptance probability among Hastings algorithms

Two complementary derivations improve understanding of the algorithm

Clarifies the conceptual basis of the Hastings algorithm

Abstract

The Hastings algorithm is a key tool in computational science. While mathematically justified by detailed balance, it can be conceptually difficult to grasp. Here, we present two complementary and intuitive ways to derive and understand the algorithm. In our framework, it is straightforward to see that the celebrated Metropolis-Hastings algorithm has the highest acceptance probability of all Hastings algorithms.