From Fields to Trees
Firas Hamze, Nando de Freitas

TL;DR
This paper introduces new MCMC algorithms for undirected graphical models, especially Markov Random Fields, by partitioning them into trees to enable exact posterior computation and improve sampling efficiency.
Contribution
It presents a novel tree-based partitioning approach for MCMC in MRFs, enabling exact solutions and more efficient sampling compared to existing methods.
Findings
Tree sampling outperforms other partitioned schemes and naive Gibbs sampling.
Tree sampling has lower variance than naive Gibbs and other naive partitioning schemes.
Theoretical analysis confirms the efficiency of tree sampling through maximal correlation and information theory tools.
Abstract
We present new MCMC algorithms for computing the posterior distributions and expectations of the unknown variables in undirected graphical models with regular structure. For demonstration purposes, we focus on Markov Random Fields (MRFs). By partitioning the MRFs into non-overlapping trees, it is possible to compute the posterior distribution of a particular tree exactly by conditioning on the remaining tree. These exact solutions allow us to construct efficient blocked and Rao-Blackwellised MCMC algorithms. We show empirically that tree sampling is considerably more efficient than other partitioned sampling schemes and the naive Gibbs sampler, even in cases where loopy belief propagation fails to converge. We prove that tree sampling exhibits lower variance than the naive Gibbs sampler and other naive partitioning schemes using the theoretical measure of maximal correlation. We also…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMarkov Chains and Monte Carlo Methods · Bayesian Modeling and Causal Inference · Bayesian Methods and Mixture Models
