Case-Factor Diagrams for Structured Probabilistic Modeling

TL;DR

This paper introduces case-factor diagrams (CFDs), a new probabilistic formalism that efficiently models structured probabilistic problems like Markov random fields and context-free grammars, enabling effective inference algorithms.

Contribution

The paper presents CFDs as a concise Boolean formula representation for structured probabilistic models, along with algorithms for marginal and MAP inference that run in linear time relative to CFD size.

Findings

CFDs efficiently represent bounded tree width Markov random fields.

CFDs can also represent parse trees for context-free grammars.

The inside-outside and Viterbi algorithms run in time proportional to CFD size.

Abstract

We introduce a probabilistic formalism subsuming Markov random fields of bounded tree width and probabilistic context free grammars. Our models are based on a representation of Boolean formulas that we call case-factor diagrams (CFDs). CFDs are similar to binary decision diagrams (BDDs) but are concise for circuits of bounded tree width (unlike BDDs) and can concisely represent the set of parse trees over a given string undera given context free grammar (also unlike BDDs). A probabilistic model consists of aCFD defining a feasible set of Boolean assignments and a weight (or cost) for each individual Boolean variable. We give an insideoutside algorithm for simultaneously computing the marginal of each Boolean variable, and a Viterbi algorithm for finding the mininum cost variable assignment. Both algorithms run in time proportional to the size of the CFD.