Learning Juntas under Markov Random Fields

TL;DR

This paper presents a polynomial-time algorithm for learning small juntas in Markov Random Fields under smoothed analysis, combining structure learning and supervised learning, advancing the understanding of graphical model learning.

Contribution

It introduces the first provably efficient supervised learning algorithm derived from structure learning of undirected graphical models in MRFs.

Findings

Successfully learns O(log n) juntas in polynomial time

Extends previous work from product distributions to MRFs with perturbed external fields

Combines structure learning with supervised learning in a novel way

Abstract

We give an algorithm for learning $O(\log n)$ juntas in polynomial-time with respect to Markov Random Fields (MRFs) in a smoothed analysis framework where only the external field has been randomly perturbed. This is a broad generalization of the work of Kalai and Teng, who gave an algorithm that succeeded with respect to smoothed product distributions (i.e., MRFs whose dependency graph has no edges). Our algorithm has two phases: (1) an unsupervised structure learning phase and (2) a greedy supervised learning algorithm. This is the first example where algorithms for learning the structure of an undirected graphical model lead to provably efficient algorithms for supervised learning.