On the trade-off between complexity and correlation decay in structural learning algorithms

TL;DR

This paper investigates the limitations of simple algorithms in learning Ising model structures, revealing that they struggle with long-range correlations near phase transitions, highlighting a trade-off between complexity and correlation decay.

Contribution

It provides a detailed analysis of how low-complexity algorithms fail in the presence of long-range correlations, clarifying their limitations near phase transitions.

Findings

Low-complexity algorithms fail with long-range correlations.

Failure correlates with the Ising model phase transition.

Trade-off exists between algorithm complexity and correlation decay.

Abstract

We consider the problem of learning the structure of Ising models (pairwise binary Markov random fields) from i.i.d. samples. While several methods have been proposed to accomplish this task, their relative merits and limitations remain somewhat obscure. By analyzing a number of concrete examples, we show that low-complexity algorithms often fail when the Markov random field develops long-range correlations. More precisely, this phenomenon appears to be related to the Ising model phase transition (although it does not coincide with it).