Efficient-NeRF2NeRF: Streamlining Text-Driven 3D Editing with Multiview Correspondence-Enhanced Diffusion Models

TL;DR

This paper introduces a method to accelerate text-driven 3D editing by incorporating multiview correspondence regularization into diffusion models, achieving a 10-fold speed-up while maintaining quality.

Contribution

It proposes a novel multiview correspondence regularization technique that significantly reduces 3D editing time in diffusion-based models.

Findings

Achieves 10x speed-up in 3D editing process.

Completes editing in approximately 2 minutes.

Maintains comparable quality to slower methods.

Abstract

The advancement of text-driven 3D content editing has been blessed by the progress from 2D generative diffusion models. However, a major obstacle hindering the widespread adoption of 3D content editing is its time-intensive processing. This challenge arises from the iterative and refining steps required to achieve consistent 3D outputs from 2D image-based generative models. Recent state-of-the-art methods typically require optimization time ranging from tens of minutes to several hours to edit a 3D scene using a single GPU. In this work, we propose that by incorporating correspondence regularization into diffusion models, the process of 3D editing can be significantly accelerated. This approach is inspired by the notion that the estimated samples during diffusion should be multiview-consistent during the diffusion generation process. By leveraging this multiview consistency, we can edit 3D content at a much faster speed. In most scenarios, our proposed technique brings a 10$\times$ speed-up compared to the baseline method and completes the editing of a 3D scene in 2 minutes with comparable quality.