Operations for Learning with Graphical Models

TL;DR

This paper reviews how graphical models serve as a unified framework for understanding and developing various learning algorithms, emphasizing decomposition techniques and graphical operations.

Contribution

It introduces decomposition techniques and demonstrates that graphical models can systematically represent and derive complex learning algorithms.

Findings

Graphical operations simplify and manipulate probability models.

Standard algorithms like Gibbs sampling and EM are expressed graphically.

Framework unifies diverse learning algorithms under a common graphical perspective.

Abstract

This paper is a multidisciplinary review of empirical, statistical learning from a graphical model perspective. Well-known examples of graphical models include Bayesian networks, directed graphs representing a Markov chain, and undirected networks representing a Markov field. These graphical models are extended to model data analysis and empirical learning using the notation of plates. Graphical operations for simplifying and manipulating a problem are provided including decomposition, differentiation, and the manipulation of probability models from the exponential family. Two standard algorithm schemas for learning are reviewed in a graphical framework: Gibbs sampling and the expectation maximization algorithm. Using these operations and schemas, some popular algorithms can be synthesized from their graphical specification. This includes versions of linear regression, techniques for feed-forward networks, and learning Gaussian and discrete Bayesian networks from data. The paper concludes by sketching some implications for data analysis and summarizing how some popular algorithms fall within the framework presented. The main original contributions here are the decomposition techniques and the demonstration that graphical models provide a framework for understanding and developing complex learning algorithms.