MCMC Learning

TL;DR

This paper explores extending learning theories from uniform distributions to Markov Random Field distributions, establishing new connections between MCMC sampling and learning in high-dimensional probabilistic models.

Contribution

It introduces a framework for applying learning methods to MRF distributions and links MCMC sampling properties to the learning process for these models.

Findings

Established a connection between MCMC sampling and learning in MRFs

Extended learning algorithms from uniform to MRF distributions

Provided theoretical foundations for learning with complex probabilistic models

Abstract

The theory of learning under the uniform distribution is rich and deep, with connections to cryptography, computational complexity, and the analysis of boolean functions to name a few areas. This theory however is very limited due to the fact that the uniform distribution and the corresponding Fourier basis are rarely encountered as a statistical model. A family of distributions that vastly generalizes the uniform distribution on the Boolean cube is that of distributions represented by Markov Random Fields (MRF). Markov Random Fields are one of the main tools for modeling high dimensional data in many areas of statistics and machine learning. In this paper we initiate the investigation of extending central ideas, methods and algorithms from the theory of learning under the uniform distribution to the setup of learning concepts given examples from MRF distributions. In particular, our results establish a novel connection between properties of MCMC sampling of MRFs and learning under the MRF distribution.