Learning Explicit Continuous Motion Representation for Dynamic Gaussian Splatting from Monocular Videos

TL;DR

This paper introduces a novel approach for dynamic Gaussian Splatting from monocular videos by explicitly modeling continuous motion using SE(3) B-spline bases, improving efficiency and accuracy in view synthesis.

Contribution

It proposes an explicit continuous motion model with adaptive control for dynamic Gaussians, enhancing modeling of complex motions and reducing overfitting in view synthesis.

Findings

Outperforms state-of-the-art in novel view synthesis

Efficient adaptive control mechanism for motion bases

Mitigates long-interval motion interference

Abstract

We present an approach for high-quality dynamic Gaussian Splatting from monocular videos. To this end, we in this work go one step further beyond previous methods to explicitly model continuous position and orientation deformation of dynamic Gaussians, using an SE(3) B-spline motion bases with a compact set of control points. To improve computational efficiency while enhancing the ability to model complex motions, an adaptive control mechanism is devised to dynamically adjust the number of motion bases and control points. Besides, we develop a soft segment reconstruction strategy to mitigate long-interval motion interference, and employ a multi-view diffusion model to provide multi-view cues for avoiding overfitting to training views. Extensive experiments demonstrate that our method outperforms state-of-the-art methods in novel view synthesis. Our code is available at https://github.com/hhhddddddd/se3bsplinegs.