Linear and Parallel Learning of Markov Random Fields

TL;DR

This paper presents a fully parallel, efficient parameter learning algorithm for Markov random fields that scales well with graph size and is data-efficient for log-linear models, improving over existing methods.

Contribution

It introduces a new embarrassingly parallel learning algorithm for Markov random fields with untied parameters, scalable to large models and requiring only local data statistics.

Findings

Algorithm is fully parallel and efficient for large models.

Complexity is linear in the number of cliques for bounded degree graphs.

Requires only local sufficient statistics for log-linear models.

Abstract

We introduce a new embarrassingly parallel parameter learning algorithm for Markov random fields with untied parameters which is efficient for a large class of practical models. Our algorithm parallelizes naturally over cliques and, for graphs of bounded degree, its complexity is linear in the number of cliques. Unlike its competitors, our algorithm is fully parallel and for log-linear models it is also data efficient, requiring only the local sufficient statistics of the data to estimate parameters.