Motion-aware 3D Gaussian Splatting for Efficient Dynamic Scene Reconstruction

TL;DR

This paper introduces a motion-aware enhancement for 3D Gaussian Splatting that leverages optical flow to improve dynamic scene reconstruction, resulting in better quality and efficiency.

Contribution

It proposes a novel framework that incorporates motion cues from optical flow into 3D Gaussian Splatting, including flow augmentation and a transient-aware deformation module.

Findings

Significantly improves rendering quality over baselines.

Enhances efficiency in dynamic scene reconstruction.

Effective on both multi-view and monocular scenes.

Abstract

3D Gaussian Splatting (3DGS) has become an emerging tool for dynamic scene reconstruction. However, existing methods focus mainly on extending static 3DGS into a time-variant representation, while overlooking the rich motion information carried by 2D observations, thus suffering from performance degradation and model redundancy. To address the above problem, we propose a novel motion-aware enhancement framework for dynamic scene reconstruction, which mines useful motion cues from optical flow to improve different paradigms of dynamic 3DGS. Specifically, we first establish a correspondence between 3D Gaussian movements and pixel-level flow. Then a novel flow augmentation method is introduced with additional insights into uncertainty and loss collaboration. Moreover, for the prevalent deformation-based paradigm that presents a harder optimization problem, a transient-aware deformation auxiliary module is proposed. We conduct extensive experiments on both multi-view and monocular scenes to verify the merits of our work. Compared with the baselines, our method shows significant superiority in both rendering quality and efficiency.