Unsupervised learning of object landmarks by factorized spatial embeddings

TL;DR

This paper introduces an unsupervised deep learning method that automatically discovers and learns meaningful object landmarks by factorizing image deformations, enabling consistent correspondence across object instances without manual annotations.

Contribution

The novel approach learns object landmarks unsupervisedly through factorizing deformations, establishing meaningful correspondences and predicting annotated landmarks with high accuracy.

Findings

Learned landmarks are consistent across object instances.

Method predicts manually-annotated landmarks with high accuracy.

Unsupervised landmarks are highly predictive of manual annotations.

Abstract

Learning automatically the structure of object categories remains an important open problem in computer vision. In this paper, we propose a novel unsupervised approach that can discover and learn landmarks in object categories, thus characterizing their structure. Our approach is based on factorizing image deformations, as induced by a viewpoint change or an object deformation, by learning a deep neural network that detects landmarks consistently with such visual effects. Furthermore, we show that the learned landmarks establish meaningful correspondences between different object instances in a category without having to impose this requirement explicitly. We assess the method qualitatively on a variety of object types, natural and man-made. We also show that our unsupervised landmarks are highly predictive of manually-annotated landmarks in face benchmark datasets, and can be used to regress these with a high degree of accuracy.

Figures

Full text

See pages 1-last of main-arxiv.pdf See pages 1-last of supplementary-arxiv.pdf