ViewNeRF: Unsupervised Viewpoint Estimation Using Category-Level Neural Radiance Fields

TL;DR

ViewNeRF is a novel unsupervised method that leverages category-level neural radiance fields to accurately estimate viewpoints from images, even in complex multi-scene scenarios, without requiring ground-truth camera poses.

Contribution

It introduces a self-supervised approach combining conditional NeRF with a viewpoint predictor and scene encoder for category-level viewpoint estimation.

Findings

Achieves accurate viewpoint prediction in complex scenarios

Performs well on synthetic and real datasets

Works on both single scenes and multi-instance collections

Abstract

We introduce ViewNeRF, a Neural Radiance Field-based viewpoint estimation method that learns to predict category-level viewpoints directly from images during training. While NeRF is usually trained with ground-truth camera poses, multiple extensions have been proposed to reduce the need for this expensive supervision. Nonetheless, most of these methods still struggle in complex settings with large camera movements, and are restricted to single scenes, i.e. they cannot be trained on a collection of scenes depicting the same object category. To address these issues, our method uses an analysis by synthesis approach, combining a conditional NeRF with a viewpoint predictor and a scene encoder in order to produce self-supervised reconstructions for whole object categories. Rather than focusing on high fidelity reconstruction, we target efficient and accurate viewpoint prediction in complex scenarios, e.g. 360{\deg} rotation on real data. Our model shows competitive results on synthetic and real datasets, both for single scenes and multi-instance collections.