EditableNeRF: Editing Topologically Varying Neural Radiance Fields by Key Points

TL;DR

EditableNeRF introduces a method for editing dynamic scenes modeled by neural radiance fields, allowing intuitive manipulation of topological changes using key points, trained from a single camera input.

Contribution

It presents a novel approach for editing topologically varying neural radiance fields through key points, enabling dynamic scene modifications from minimal input data.

Findings

Supports multi-dimensional editing up to 3D

Achieves high-quality scene editing results

Outperforms state-of-the-art methods in dynamic scene editing

Abstract

Neural radiance fields (NeRF) achieve highly photo-realistic novel-view synthesis, but it's a challenging problem to edit the scenes modeled by NeRF-based methods, especially for dynamic scenes. We propose editable neural radiance fields that enable end-users to easily edit dynamic scenes and even support topological changes. Input with an image sequence from a single camera, our network is trained fully automatically and models topologically varying dynamics using our picked-out surface key points. Then end-users can edit the scene by easily dragging the key points to desired new positions. To achieve this, we propose a scene analysis method to detect and initialize key points by considering the dynamics in the scene, and a weighted key points strategy to model topologically varying dynamics by joint key points and weights optimization. Our method supports intuitive multi-dimensional (up to 3D) editing and can generate novel scenes that are unseen in the input sequence. Experiments demonstrate that our method achieves high-quality editing on various dynamic scenes and outperforms the state-of-the-art. Our code and captured data are available at https://chengwei-zheng.github.io/EditableNeRF/.