Scene Grammars, Factor Graphs, and Belief Propagation

TL;DR

This paper introduces a probabilistic framework using scene grammars and factor graphs for complex scene modeling and inference, demonstrated through image analysis tasks like contour reconstruction and face detection.

Contribution

It presents a novel class of stochastic scene grammars integrated with graphical models for efficient inference via belief propagation.

Findings

Effective scene modeling with probabilistic grammars

Robust inference in image analysis applications

Successful application to contour and face detection

Abstract

We describe a general framework for probabilistic modeling of complex scenes and inference from ambiguous observations. The approach is motivated by applications in image analysis and is based on the use of priors defined by stochastic grammars. We define a class of grammars that capture relationships between the objects in a scene and provide important contextual cues for statistical inference. The distribution over scenes defined by a probabilistic scene grammar can be represented by a graphical model and this construction can be used for efficient inference with loopy belief propagation. We show experimental results with two different applications. One application involves the reconstruction of binary contour maps. Another application involves detecting and localizing faces in images. In both applications the same framework leads to robust inference algorithms that can effectively combine local information to reason about a scene.