LOLNeRF: Learn from One Look

TL;DR

This paper introduces LOLNeRF, a method for learning 3D generative models from single-view data, enabling realistic rendering and shape reconstruction without multi-view supervision.

Contribution

It presents a novel approach that trains a neural radiance field model using only single-view images, eliminating the need for multi-view data or depth information.

Findings

Achieves state-of-the-art novel view synthesis results.

Produces high-quality monocular depth predictions.

Successfully models shape and appearance from single-view datasets.

Abstract

We present a method for learning a generative 3D model based on neural radiance fields, trained solely from data with only single views of each object. While generating realistic images is no longer a difficult task, producing the corresponding 3D structure such that they can be rendered from different views is non-trivial. We show that, unlike existing methods, one does not need multi-view data to achieve this goal. Specifically, we show that by reconstructing many images aligned to an approximate canonical pose with a single network conditioned on a shared latent space, you can learn a space of radiance fields that models shape and appearance for a class of objects. We demonstrate this by training models to reconstruct object categories using datasets that contain only one view of each subject without depth or geometry information. Our experiments show that we achieve state-of-the-art results in novel view synthesis and high-quality results for monocular depth prediction.