EMD: Explicit Motion Modeling for High-Quality Street Gaussian Splatting

TL;DR

This paper introduces Explicit Motion Decomposition (EMD), a novel method that models dynamic object motions in street scenes using learnable embeddings, significantly improving Gaussian Splatting for photorealistic scene reconstruction.

Contribution

The paper proposes a plug-and-play EMD module with tailored training strategies, enhancing motion modeling in street scene Gaussian Splatting, and achieves state-of-the-art results in self-supervised view synthesis.

Findings

EMD effectively models dynamic object motions in street scenes.

The method achieves superior novel view synthesis performance.

It enhances scene decomposition accuracy in complex environments.

Abstract

Photorealistic reconstruction of street scenes is essential for developing real-world simulators in autonomous driving. While recent methods based on 3D/4D Gaussian Splatting (GS) have demonstrated promising results, they still encounter challenges in complex street scenes due to the unpredictable motion of dynamic objects. Current methods typically decompose street scenes into static and dynamic objects, learning the Gaussians in either a supervised manner (e.g., w/ 3D bounding-box) or a self-supervised manner (e.g., w/o 3D bounding-box). However, these approaches do not effectively model the motions of dynamic objects (e.g., the motion speed of pedestrians is clearly different from that of vehicles), resulting in suboptimal scene decomposition. To address this, we propose Explicit Motion Decomposition (EMD), which models the motions of dynamic objects by introducing learnable motion embeddings to the Gaussians, enhancing the decomposition in street scenes. The proposed plug-and-play EMD module compensates for the lack of motion modeling in self-supervised street Gaussian splatting methods. We also introduce tailored training strategies to extend EMD to supervised approaches. Comprehensive experiments demonstrate the effectiveness of our method, achieving state-of-the-art novel view synthesis performance in self-supervised settings. The code is available at: https://qingpowuwu.github.io/emd.