Instruct-4DGS: Efficient Dynamic Scene Editing via 4D Gaussian-based Static-Dynamic Separation

TL;DR

Instruct-4DGS introduces a scalable, efficient method for dynamic scene editing that leverages 4D Gaussian representations and a refinement stage, significantly reducing editing time while maintaining high-quality, instruction-following edits.

Contribution

The paper presents a novel 4D Gaussian-based approach for dynamic scene editing that improves scalability and efficiency over existing methods by focusing editing on static components and refining alignment.

Findings

Reduces editing time by over 50% compared to prior methods.

Achieves high-quality edits that accurately follow user instructions.

Demonstrates scalability to scenes with many temporal steps.

Abstract

Recent 4D dynamic scene editing methods require editing thousands of 2D images used for dynamic scene synthesis and updating the entire scene with additional training loops, resulting in several hours of processing to edit a single dynamic scene. Therefore, these methods are not scalable with respect to the temporal dimension of the dynamic scene (i.e., the number of timesteps). In this work, we propose Instruct-4DGS, an efficient dynamic scene editing method that is more scalable in terms of temporal dimension. To achieve computational efficiency, we leverage a 4D Gaussian representation that models a 4D dynamic scene by combining static 3D Gaussians with a Hexplane-based deformation field, which captures dynamic information. We then perform editing solely on the static 3D Gaussians, which is the minimal but sufficient component required for visual editing. To resolve the misalignment between the edited 3D Gaussians and the deformation field, which may arise from the editing process, we introduce a refinement stage using a score distillation mechanism. Extensive editing results demonstrate that Instruct-4DGS is efficient, reducing editing time by more than half compared to existing methods while achieving high-quality edits that better follow user instructions. Code and results: https://hanbyelcho.info/instruct-4dgs/