Recoverability Landscape of Tree Structured Markov Random Fields under Symmetric Noise

TL;DR

This paper investigates the recoverability of tree-structured Markov random fields with support size three or more under symmetric noise, providing conditions for exact recovery and an efficient algorithm, extending prior work beyond Ising models.

Contribution

It characterizes when the structure of tree-structured MRFs with larger support sizes can be exactly recovered under noise, and introduces a polynomial-time algorithm for this task.

Findings

Exact recoverability depends on the joint PMF of variables.

Provided necessary and sufficient conditions for structure recovery.

Developed an efficient algorithm that achieves optimal recovery.

Abstract

We study the problem of learning tree-structured Markov random fields (MRF) on discrete random variables with common support when the observations are corrupted by a $k$-ary symmetric noise channel with unknown probability of error. For Ising models (support size = 2), past work has shown that graph structure can only be recovered up to the leaf clusters (a leaf node, its parent, and its siblings form a leaf cluster) and exact recovery is impossible. No prior work has addressed the setting of support size of 3 or more, and indeed this setting is far richer. As we show, when the support size is 3 or more, the structure of the leaf clusters may be partially or fully identifiable. We provide a precise characterization of this phenomenon and show that the extent of recoverability is dictated by the joint PMF of the random variables. In particular, we provide necessary and sufficient conditions for exact recoverability. Furthermore, we present a polynomial time, sample efficient algorithm that recovers the exact tree when this is possible, or up to the unidentifiability as promised by our characterization, when full recoverability is impossible. Finally, we demonstrate the efficacy of our algorithm experimentally.