ProGDF: Progressive Gaussian Differential Field for Controllable and Flexible 3D Editing

TL;DR

ProGDF introduces a real-time, process-oriented 3D editing method that enables controllable and flexible modifications with a single training session, improving user experience and editing efficiency.

Contribution

The paper presents ProGDF, a novel out-of-loop training approach combining Progressive Gaussian Splatting and Gaussian Differential Field for efficient, controllable 3D editing.

Findings

Enables real-time 3D editing with a single training session

Demonstrates high-quality, controllable, and flexible 3D manipulation

Outperforms existing methods in efficiency and user experience

Abstract

3D editing plays a crucial role in editing and reusing existing 3D assets, thereby enhancing productivity. Recently, 3DGS-based methods have gained increasing attention due to their efficient rendering and flexibility. However, achieving desired 3D editing results often requires multiple adjustments in an iterative loop, resulting in tens of minutes of training time cost for each attempt and a cumbersome trial-and-error cycle for users. This in-the-loop training paradigm results in a poor user experience. To address this issue, we introduce the concept of process-oriented modelling for 3D editing and propose the Progressive Gaussian Differential Field (ProGDF), an out-of-loop training approach that requires only a single training session to provide users with controllable editing capability and variable editing results through a user-friendly interface in real-time. ProGDF consists of two key components: Progressive Gaussian Splatting (PGS) and Gaussian Differential Field (GDF). PGS introduces the progressive constraint to extract the diverse intermediate results of the editing process and employs rendering quality regularization to improve the quality of these results. Based on these intermediate results, GDF leverages a lightweight neural network to model the editing process. Extensive results on two novel applications, namely controllable 3D editing and flexible fine-grained 3D manipulation, demonstrate the effectiveness, practicality and flexibility of the proposed ProGDF.