DualNeRF: Text-Driven 3D Scene Editing via Dual-Field Representation

TL;DR

DualNeRF introduces a dual-field representation and a simulated annealing strategy to enhance 3D scene editing, effectively preserving backgrounds and avoiding local optima, resulting in superior editing quality.

Contribution

The paper presents a novel dual-field representation and annealing strategy to improve 3D scene editing, addressing background preservation and local optima issues in diffusion-based methods.

Findings

Outperforms previous methods qualitatively and quantitatively

Effectively preserves scene backgrounds during editing

Reduces local optima trapping in 3D scene editing

Abstract

Recently, denoising diffusion models have achieved promising results in 2D image generation and editing. Instruct-NeRF2NeRF (IN2N) introduces the success of diffusion into 3D scene editing through an "Iterative dataset update" (IDU) strategy. Though achieving fascinating results, IN2N suffers from problems of blurry backgrounds and trapping in local optima. The first problem is caused by IN2N's lack of efficient guidance for background maintenance, while the second stems from the interaction between image editing and NeRF training during IDU. In this work, we introduce DualNeRF to deal with these problems. We propose a dual-field representation to preserve features of the original scene and utilize them as additional guidance to the model for background maintenance during IDU. Moreover, a simulated annealing strategy is embedded into IDU to endow our model with the power of addressing local optima issues. A CLIP-based consistency indicator is used to further improve the editing quality by filtering out low-quality edits. Extensive experiments demonstrate that our method outperforms previous methods both qualitatively and quantitatively.