Bayesian learning of forest and tree graphical models

TL;DR

This paper advances Bayesian methods for learning forest and tree structures in Gaussian graphical models, introducing corrected algorithms, efficient search techniques, and demonstrating their effectiveness in sparse and tree-like graphs.

Contribution

It presents corrected algorithms for non-decomposable graphs, adapts MCMC and SSS methods for forests and trees, and evaluates their performance in structure learning.

Findings

SSS with trees performs well on true tree or sparse graphs

Forests and trees improve over decomposable graphs in certain cases

Graph priors enhance hub detection but require broad probability ranges

Abstract

In Bayesian learning of Gaussian graphical model structure, it is common to restrict attention to certain classes of graphs and approximate the posterior distribution by repeatedly moving from one graph to another, using MCMC or methods such as stochastic shotgun search (SSS). I give two corrected versions of an algorithm for non-decomposable graphs and discuss random graph distributions, in particular as prior distributions. The main topic of the thesis is Bayesian structure-learning with forests or trees. Restricting attention to these graphs can be justified using theorems on random graphs. I describe how to use the Chow$\unicode{x2013}$Liu algorithm and the Matrix Tree Theorem to find the MAP forest and certain quantities in the posterior distribution on trees. I give adapted versions of MCMC and SSS for approximating the posterior distribution for forests and trees, and systems for storing these graphs so that it is easy to choose moves to neighbouring graphs. Experiments show that SSS with trees does well when the true graph is a tree or sparse graph. SSS with trees or forests does better than SSS with decomposable graphs in certain cases. Graph priors improve detection of hubs but need large ranges of probabilities. MCMC on forests fails to mix well and MCMC on trees is slower than SSS. (For a longer abstract see the thesis.)