Removing Objects From Neural Radiance Fields

TL;DR

This paper introduces a novel method for removing objects from Neural Radiance Fields (NeRFs) by leveraging 2D inpainting guided by user masks, ensuring 3D consistency and multi-view coherence.

Contribution

The authors propose a NeRF editing framework that uses confidence-based view selection and 2D inpainting to effectively remove objects while maintaining 3D consistency.

Findings

Effective object removal in NeRFs with multi-view coherence

Uses confidence-based view selection for better inpainting results

Validated on a new challenging NeRF inpainting dataset

Abstract

Neural Radiance Fields (NeRFs) are emerging as a ubiquitous scene representation that allows for novel view synthesis. Increasingly, NeRFs will be shareable with other people. Before sharing a NeRF, though, it might be desirable to remove personal information or unsightly objects. Such removal is not easily achieved with the current NeRF editing frameworks. We propose a framework to remove objects from a NeRF representation created from an RGB-D sequence. Our NeRF inpainting method leverages recent work in 2D image inpainting and is guided by a user-provided mask. Our algorithm is underpinned by a confidence based view selection procedure. It chooses which of the individual 2D inpainted images to use in the creation of the NeRF, so that the resulting inpainted NeRF is 3D consistent. We show that our method for NeRF editing is effective for synthesizing plausible inpaintings in a multi-view coherent manner. We validate our approach using a new and still-challenging dataset for the task of NeRF inpainting.