Learning Graphical Models of Images, Videos and Their Spatial Transformations

TL;DR

This paper introduces transformation-invariant graphical models for images and videos by incorporating discrete transformation variables, enabling robust clustering, dimensionality reduction, and analysis despite spatial transformations.

Contribution

It presents a novel approach to modeling spatial transformations within graphical models using a discrete variable, enhancing their ability to handle transformed data.

Findings

Effective clustering of faces and facial poses

Successful recognition of handwritten digits

Robust video clustering and object tracking

Abstract

Mixtures of Gaussians, factor analyzers (probabilistic PCA) and hidden Markov models are staples of static and dynamic data modeling and image and video modeling in particular. We show how topographic transformations in the input, such as translation and shearing in images, can be accounted for in these models by including a discrete transformation variable. The resulting models perform clustering, dimensionality reduction and time-series analysis in a way that is invariant to transformations in the input. Using the EM algorithm, these transformation-invariant models can be fit to static data and time series. We give results on filtering microscopy images, face and facial pose clustering, handwritten digit modeling and recognition, video clustering, object tracking, and removal of distractions from video sequences.