Perturb-and-Revise: Flexible 3D Editing with Generative Trajectories

TL;DR

Perturb-and-Revise introduces a novel NeRF editing method that uses parameter perturbation and generative trajectories to enable flexible, consistent 3D editing of color, appearance, and geometry, overcoming limitations of existing techniques.

Contribution

It proposes a new NeRF editing framework combining perturbation and generative trajectories for enhanced 3D editing capabilities.

Findings

Enables extensive geometric and appearance modifications.

Demonstrates improved consistency and effectiveness in 3D editing.

Supports high-quality 360-degree results.

Abstract

Recent advancements in text-based diffusion models have accelerated progress in 3D reconstruction and text-based 3D editing. Although existing 3D editing methods excel at modifying color, texture, and style, they struggle with extensive geometric or appearance changes, thus limiting their applications. To this end, we propose Perturb-and-Revise, which makes possible a variety of NeRF editing. First, we perturb the NeRF parameters with random initializations to create a versatile initialization. The level of perturbation is determined automatically through analysis of the local loss landscape. Then, we revise the edited NeRF via generative trajectories. Combined with the generative process, we impose identity-preserving gradients to refine the edited NeRF. Extensive experiments demonstrate that Perturb-and-Revise facilitates flexible, effective, and consistent editing of color, appearance, and geometry in 3D. For 360{\deg} results, please visit our project page: https://susunghong.github.io/Perturb-and-Revise.