Hamiltonian Annealed Importance Sampling for partition function estimation

TL;DR

This paper presents a Hamiltonian-based extension to annealed importance sampling for efficient partition function estimation, demonstrated on various probabilistic image models with improved performance.

Contribution

It introduces a novel Hamiltonian annealed importance sampling method for faster and more accurate normalization constant estimation in probabilistic models.

Findings

Effective estimation of log likelihoods in image models

Comparison shows advantages over traditional methods

Provides code for broader model evaluation

Abstract

We introduce an extension to annealed importance sampling that uses Hamiltonian dynamics to rapidly estimate normalization constants. We demonstrate this method by computing log likelihoods in directed and undirected probabilistic image models. We compare the performance of linear generative models with both Gaussian and Laplace priors, product of experts models with Laplace and Student's t experts, the mc-RBM, and a bilinear generative model. We provide code to compare additional models.