Drag Your Gaussian: Effective Drag-Based Editing with Score Distillation for 3D Gaussian Splatting

TL;DR

DYG introduces a precise, drag-based 3D editing method for Gaussian Splatting that allows users to control editing regions and directions, overcoming previous texture-only limitations and improving geometric editing accuracy.

Contribution

The paper presents DYG, a novel 3D drag-based editing approach integrating control point prompts and an implicit triplane scaffold for enhanced geometric and multi-view consistent editing.

Findings

Outperforms baselines in editing quality and effect

Enables precise control over editing regions and directions

Supports flexible, multi-view consistent 3D edits

Abstract

Recent advancements in 3D scene editing have been propelled by the rapid development of generative models. Existing methods typically utilize generative models to perform text-guided editing on 3D representations, such as 3D Gaussian Splatting (3DGS). However, these methods are often limited to texture modifications and fail when addressing geometric changes, such as editing a character's head to turn around. Moreover, such methods lack accurate control over the spatial position of editing results, as language struggles to precisely describe the extent of edits. To overcome these limitations, we introduce DYG, an effective 3D drag-based editing method for 3D Gaussian Splatting. It enables users to conveniently specify the desired editing region and the desired dragging direction through the input of 3D masks and pairs of control points, thereby enabling precise control over the extent of editing. DYG integrates the strengths of the implicit triplane representation to establish the geometric scaffold of the editing results, effectively overcoming suboptimal editing outcomes caused by the sparsity of 3DGS in the desired editing regions. Additionally, we incorporate a drag-based Latent Diffusion Model into our method through the proposed Drag-SDS loss function, enabling flexible, multi-view consistent, and fine-grained editing. Extensive experiments demonstrate that DYG conducts effective drag-based editing guided by control point prompts, surpassing other baselines in terms of editing effect and quality, both qualitatively and quantitatively. Visit our project page at https://quyans.github.io/Drag-Your-Gaussian.