Conjugate Generalised Bayesian Inference for Discrete Doubly Intractable Problems

TL;DR

This paper introduces a new conjugate Bayesian inference method for discrete doubly intractable problems, offering a computationally efficient alternative to traditional MCMC techniques, with strong theoretical guarantees and significant speed improvements.

Contribution

It develops a novel generalized Bayesian posterior enabling conjugate inference for exponential family models in discrete data, reducing computational costs.

Findings

Method is 10 to 6000 times faster than existing approaches.

Theoretical guarantees support the asymptotic validity of the generalized posterior.

Effective on complex models like Markov random fields and count data models.

Abstract

Doubly intractable problems occur when both the likelihood and the posterior are available only in unnormalised form, with computationally intractable normalisation constants. Bayesian inference then typically requires direct approximation of the posterior through specialised and typically expensive MCMC methods. In this paper, we provide a computationally efficient alternative in the form of a novel generalised Bayesian posterior that allows for conjugate inference within the class of exponential family models for discrete data. We derive theoretical guarantees to characterise the asymptotic behaviour of the generalised posterior, supporting its use for inference. The method is evaluated on a range of challenging intractable exponential family models, including the Conway-Maxwell-Poisson graphical model of multivariate count data, autoregressive discrete time series models, and Markov random fields such as the Ising and Potts models. The computational gains are significant; in our experiments, the method is between 10 and 6000 times faster than state-of-the-art Bayesian computational methods.