Unsupervised Learning of Compositional Scene Representations from Multiple Unspecified Viewpoints

TL;DR

This paper introduces a deep generative model that learns to represent complex scenes compositionally from multiple, unspecified viewpoints without supervision, mimicking human perception of object constancy across viewpoints.

Contribution

It proposes a novel unsupervised model that separates viewpoint-independent and viewpoint-dependent features, enabling effective learning from multiple viewpoints.

Findings

Successfully learns scene representations from synthetic datasets

Achieves object constancy across different viewpoints

Demonstrates effectiveness of the proposed model in unsupervised settings

Abstract

Visual scenes are extremely rich in diversity, not only because there are infinite combinations of objects and background, but also because the observations of the same scene may vary greatly with the change of viewpoints. When observing a visual scene that contains multiple objects from multiple viewpoints, humans are able to perceive the scene in a compositional way from each viewpoint, while achieving the so-called "object constancy" across different viewpoints, even though the exact viewpoints are untold. This ability is essential for humans to identify the same object while moving and to learn from vision efficiently. It is intriguing to design models that have the similar ability. In this paper, we consider a novel problem of learning compositional scene representations from multiple unspecified viewpoints without using any supervision, and propose a deep generative model which separates latent representations into a viewpoint-independent part and a viewpoint-dependent part to solve this problem. To infer latent representations, the information contained in different viewpoints is iteratively integrated by neural networks. Experiments on several specifically designed synthetic datasets have shown that the proposed method is able to effectively learn from multiple unspecified viewpoints.