Structure Learning of Markov Random Fields through Grow-Shrink Maximum Pseudolikelihood Estimation

TL;DR

This paper introduces a grow-shrink maximum pseudolikelihood estimation method for learning Markov random field structures, effectively handling symmetricity and asymmetric dependencies, and outperforming previous methods in accuracy.

Contribution

It explicitly formulates MRF structure learning as maximum pseudolikelihood estimation, addressing symmetricity issues and improving accuracy over existing independence test methods.

Findings

Higher accuracy in asymmetric dependency scenarios

Effective handling of symmetricity in MRFs

Outperforms previous independence test methods

Abstract

Learning the structure of Markov random fields (MRFs) plays an important role in multivariate analysis. The importance has been increasing with the recent rise of statistical relational models since the MRF serves as a building block of these models such as Markov logic networks. There are two fundamental ways to learn structures of MRFs: methods based on parameter learning and those based on independence test. The former methods more or less assume certain forms of distribution, so they potentially perform poorly when the assumption is not satisfied. The latter can learn an MRF structure without a strong distributional assumption, but sometimes it is unclear what objective function is maximized/minimized in these methods. In this paper, we follow the latter, but we explicitly define the optimization problem of MRF structure learning as maximum pseudolikelihood estimation (MPLE) with respect to the edge set. As a result, the proposed solution successfully deals with the {\em symmetricity} in MRFs, whereas such symmetricity is not taken into account in most existing independence test techniques. The proposed method achieved higher accuracy than previous methods when there were asymmetric dependencies in our experiments.