Which graphical models are difficult to learn?

TL;DR

This paper investigates the challenges in learning the structure of Ising models, revealing that low-complexity algorithms struggle with long-range correlations, especially near phase transitions.

Contribution

The paper provides a detailed analysis of the limitations of existing structure learning algorithms for Ising models, especially under long-range correlations.

Findings

Low-complexity algorithms fail with long-range correlations.

Failure is linked to the Ising model phase transition.

Challenges are not solely due to phase transition but related to correlation length.

Abstract

We consider the problem of learning the structure of Ising models (pairwise binary Markov random fields) from i.i.d. samples. While several methods have been proposed to accomplish this task, their relative merits and limitations remain somewhat obscure. By analyzing a number of concrete examples, we show that low-complexity algorithms systematically fail when the Markov random field develops long-range correlations. More precisely, this phenomenon appears to be related to the Ising model phase transition (although it does not coincide with it).