FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modelling

TL;DR

FiG-NeRF introduces a novel 2-component neural radiance field model that effectively learns 3D object categories from casual images while separating foreground objects from backgrounds, enabling high-quality view synthesis and segmentation.

Contribution

The paper presents FiG-NeRF, a new model that jointly learns 3D object categories and separates foreground from background using only photometric supervision.

Findings

Outperforms existing methods in view synthesis and image fidelity

Achieves accurate amodal segmentation of objects

Works with synthetic, lab-captured, and in-the-wild data

Abstract

We investigate the use of Neural Radiance Fields (NeRF) to learn high quality 3D object category models from collections of input images. In contrast to previous work, we are able to do this whilst simultaneously separating foreground objects from their varying backgrounds. We achieve this via a 2-component NeRF model, FiG-NeRF, that prefers explanation of the scene as a geometrically constant background and a deformable foreground that represents the object category. We show that this method can learn accurate 3D object category models using only photometric supervision and casually captured images of the objects. Additionally, our 2-part decomposition allows the model to perform accurate and crisp amodal segmentation. We quantitatively evaluate our method with view synthesis and image fidelity metrics, using synthetic, lab-captured, and in-the-wild data. Our results demonstrate convincing 3D object category modelling that exceed the performance of existing methods.