Neural Impostor: Editing Neural Radiance Fields with Explicit Shape Manipulation

TL;DR

Neural Impostor introduces a hybrid explicit-implicit representation for efficient and controllable editing of Neural Radiance Fields, enabling shape manipulation and complex volumetric editing for 3D scene generation.

Contribution

The paper presents a novel hybrid explicit-implicit framework with a pipeline for geometric editing of NeRFs, improving flexibility and control in 3D scene editing.

Findings

Effective shape editing demonstrated on synthetic and real data

Robust geometric manipulation with explicit mesh and implicit field integration

Enables complex volumetric editing and hybrid object creation

Abstract

Neural Radiance Fields (NeRF) have significantly advanced the generation of highly realistic and expressive 3D scenes. However, the task of editing NeRF, particularly in terms of geometry modification, poses a significant challenge. This issue has obstructed NeRF's wider adoption across various applications. To tackle the problem of efficiently editing neural implicit fields, we introduce Neural Impostor, a hybrid representation incorporating an explicit tetrahedral mesh alongside a multigrid implicit field designated for each tetrahedron within the explicit mesh. Our framework bridges the explicit shape manipulation and the geometric editing of implicit fields by utilizing multigrid barycentric coordinate encoding, thus offering a pragmatic solution to deform, composite, and generate neural implicit fields while maintaining a complex volumetric appearance. Furthermore, we propose a comprehensive pipeline for editing neural implicit fields based on a set of explicit geometric editing operations. We show the robustness and adaptability of our system through diverse examples and experiments, including the editing of both synthetic objects and real captured data. Finally, we demonstrate the authoring process of a hybrid synthetic-captured object utilizing a variety of editing operations, underlining the transformative potential of Neural Impostor in the field of 3D content creation and manipulation.