Efficient inference in occlusion-aware generative models of images

TL;DR

This paper introduces a layered generative model for images that can factor object appearance, shape, and pose without labeled data, using differentiable inference to effectively separate foreground and background objects.

Contribution

It presents a novel layered generative model with shape priors that enables unsupervised inference of object occlusion, shape, and pose from images.

Findings

Effective separation of foreground and background objects

Unsupervised learning of shape priors for occlusion reasoning

Successful inference of object pose and appearance

Abstract

We present a generative model of images based on layering, in which image layers are individually generated, then composited from front to back. We are thus able to factor the appearance of an image into the appearance of individual objects within the image --- and additionally for each individual object, we can factor content from pose. Unlike prior work on layered models, we learn a shape prior for each object/layer, allowing the model to tease out which object is in front by looking for a consistent shape, without needing access to motion cues or any labeled data. We show that ordinary stochastic gradient variational bayes (SGVB), which optimizes our fully differentiable lower-bound on the log-likelihood, is sufficient to learn an interpretable representation of images. Finally we present experiments demonstrating the effectiveness of the model for inferring foreground and background objects in images.