Similarity-Driven Proposals for MCMC Algorithms on Discrete Spaces

TL;DR

This paper introduces a novel similarity-driven MCMC algorithm for discrete spaces that leverages data-driven discrepancy measures, effectively handling hierarchical models with discrete and latent variables.

Contribution

It proposes a new MCMC methodology based on similarity-driven proposals that improve sampling efficiency for discrete and hierarchical models without requiring integration of latent variables.

Findings

Effective in simulation settings

Successfully applied to real Dirichlet-Multinomial regression data

Handles hierarchical models with discrete and latent variables

Abstract

Recent research has led to the development of MCMC algorithms with likelihood-informed proposals when targeting posterior distributions supported on discrete state spaces. Our work is placed within this field and puts forward a new MCMC methodology based upon similarity-driven proposals. Such proposals sway transitions towards states favored by the posterior via use of a data-driven measure of discrepancy between observations and the proposed model. Our approach can naturally cover classes of hierarchical models that involve both discrete variables and additional latent ones, without a requirement of integrating our the latter, in contrast to previous works in this field. The new algorithms are illustrated in simulation settings and in a involved real data scenario with a Dirichlet-Multinomial regression model.