Markov Network Structure Learning via Ensemble-of-Forests Models

TL;DR

This paper introduces the ensemble-of-forests model for learning the structure of Markov random fields, capable of handling complex, disconnected, and non-Gaussian dependencies in high-dimensional data.

Contribution

It generalizes the ensemble-of-trees model to enable structure learning of MRFs with multiple components and arbitrary potentials, along with two approximate inference methods.

Findings

Accurately recovers sparse, disconnected MRF topologies

Performs well with non-Gaussian dependencies and low sample sizes

Successfully applied to immune cell signaling networks

Abstract

Real world systems typically feature a variety of different dependency types and topologies that complicate model selection for probabilistic graphical models. We introduce the ensemble-of-forests model, a generalization of the ensemble-of-trees model. Our model enables structure learning of Markov random fields (MRF) with multiple connected components and arbitrary potentials. We present two approximate inference techniques for this model and demonstrate their performance on synthetic data. Our results suggest that the ensemble-of-forests approach can accurately recover sparse, possibly disconnected MRF topologies, even in presence of non-Gaussian dependencies and/or low sample size. We applied the ensemble-of-forests model to learn the structure of perturbed signaling networks of immune cells and found that these frequently exhibit non-Gaussian dependencies with disconnected MRF topologies. In summary, we expect that the ensemble-of-forests model will enable MRF structure learning in other high dimensional real world settings that are governed by non-trivial dependencies.