TrAME: Trajectory-Anchored Multi-View Editing for Text-Guided 3D Gaussian Splatting Manipulation

TL;DR

This paper introduces TrAME, a novel 3D editing method that maintains multi-view consistency through a trajectory-anchored scheme and view-aware attention, significantly improving text-guided 3D scene editing quality.

Contribution

We propose a trajectory-anchored scheme with a dual-branch editing mechanism and a tuning-free attention module, advancing multi-view consistency in text-guided 3D scene editing.

Findings

Enhanced multi-view consistency demonstrated in 2D and 3D examples

Superior editing quality compared to state-of-the-art methods

Effective integration of cross-view semantic and geometric references

Abstract

Despite significant strides in the field of 3D scene editing, current methods encounter substantial challenge, particularly in preserving 3D consistency in multi-view editing process. To tackle this challenge, we propose a progressive 3D editing strategy that ensures multi-view consistency via a Trajectory-Anchored Scheme (TAS) with a dual-branch editing mechanism. Specifically, TAS facilitates a tightly coupled iterative process between 2D view editing and 3D updating, preventing error accumulation yielded from text-to-image process. Additionally, we explore the relationship between optimization-based methods and reconstruction-based methods, offering a unified perspective for selecting superior design choice, supporting the rationale behind the designed TAS. We further present a tuning-free View-Consistent Attention Control (VCAC) module that leverages cross-view semantic and geometric reference from the source branch to yield aligned views from the target branch during the editing of 2D views. To validate the effectiveness of our method, we analyze 2D examples to demonstrate the improved consistency with the VCAC module. Further extensive quantitative and qualitative results in text-guided 3D scene editing indicate that our method achieves superior editing quality compared to state-of-the-art methods. We will make the complete codebase publicly available following the conclusion of the review process.