Bayesian Parameter Estimation for Latent Markov Random Fields and Social Networks

TL;DR

This paper compares two Bayesian methods for estimating parameters in latent Markov random fields and social networks from noisy or incomplete data, addressing the challenge of intractable normalising constants.

Contribution

It introduces a novel combination of particle MCMC and the exchange algorithm for efficient Bayesian estimation in complex undirected models with noisy data.

Findings

Particle MCMC combined with the exchange algorithm effectively estimates parameters.

Approximate Bayesian computation provides an alternative approach.

Applications demonstrate the methods on Ising models and exponential random graphs.

Abstract

Undirected graphical models are widely used in statistics, physics and machine vision. However Bayesian parameter estimation for undirected models is extremely challenging, since evaluation of the posterior typically involves the calculation of an intractable normalising constant. This problem has received much attention, but very little of this has focussed on the important practical case where the data consists of noisy or incomplete observations of the underlying hidden structure. This paper specifically addresses this problem, comparing two alternative methodologies. In the first of these approaches particle Markov chain Monte Carlo (Andrieu et al., 2010) is used to efficiently explore the parameter space, combined with the exchange algorithm (Murray et al., 2006) for avoiding the calculation of the intractable normalising constant (a proof showing that this combination targets the correct distribution in found in a supplementary appendix online). This approach is compared with approximate Bayesian computation (Pritchard et al., 1999). Applications to estimating the parameters of Ising models and exponential random graphs from noisy data are presented. Each algorithm used in the paper targets an approximation to the true posterior due to the use of MCMC to simulate from the latent graphical model, in lieu of being able to do this exactly in general. The supplementary appendix also describes the nature of the resulting approximation.