Differentiating Metropolis-Hastings to Optimize Intractable Densities

TL;DR

This paper introduces an automatic differentiation method for Metropolis-Hastings algorithms, enabling gradient-based optimization of models with intractable densities, including those with discrete components.

Contribution

It combines stochastic automatic differentiation with Markov chain coupling to produce unbiased, low-variance gradients for intractable probabilistic models.

Findings

Successfully identified ambiguous observations in Gaussian mixture models

Maximized specific heat in Ising models using gradient-based methods

Demonstrated applicability to models with discrete components

Abstract

We develop an algorithm for automatic differentiation of Metropolis-Hastings samplers, allowing us to differentiate through probabilistic inference, even if the model has discrete components within it. Our approach fuses recent advances in stochastic automatic differentiation with traditional Markov chain coupling schemes, providing an unbiased and low-variance gradient estimator. This allows us to apply gradient-based optimization to objectives expressed as expectations over intractable target densities. We demonstrate our approach by finding an ambiguous observation in a Gaussian mixture model and by maximizing the specific heat in an Ising model.