ViewNet: Unsupervised Viewpoint Estimation from Conditional Generation

TL;DR

ViewNet introduces an unsupervised method for estimating camera viewpoints by leveraging image pairs and self-supervised learning, enabling effective 3D understanding without costly annotations.

Contribution

It proposes a novel self-supervised approach using image pairs and a perspective spatial transformer for viewpoint estimation, outperforming prior unsupervised methods.

Findings

Outperforms existing unsupervised methods on synthetic data

Achieves competitive results on PASCAL3D+ dataset

Demonstrates effective viewpoint learning without supervision

Abstract

Understanding the 3D world without supervision is currently a major challenge in computer vision as the annotations required to supervise deep networks for tasks in this domain are expensive to obtain on a large scale. In this paper, we address the problem of unsupervised viewpoint estimation. We formulate this as a self-supervised learning task, where image reconstruction provides the supervision needed to predict the camera viewpoint. Specifically, we make use of pairs of images of the same object at training time, from unknown viewpoints, to self-supervise training by combining the viewpoint information from one image with the appearance information from the other. We demonstrate that using a perspective spatial transformer allows efficient viewpoint learning, outperforming existing unsupervised approaches on synthetic data, and obtains competitive results on the challenging PASCAL3D+ dataset.