Gaussian Splatting Lucas-Kanade

TL;DR

This paper introduces an analytical extension of the Lucas-Kanade method for dynamic Gaussian splatting, enabling accurate scene flow computation and improved reconstruction of highly dynamic scenes with minimal camera movement.

Contribution

It proposes a novel analytical approach that leverages the properties of the forward warp field network to improve scene flow estimation in dynamic Gaussian splatting.

Findings

Effective in reconstructing highly dynamic scenes

Performs well on synthetic and real-world data

Enhances scene flow accuracy with minimal camera movement

Abstract

Gaussian Splatting and its dynamic extensions are effective for reconstructing 3D scenes from 2D images when there is significant camera movement to facilitate motion parallax and when scene objects remain relatively static. However, in many real-world scenarios, these conditions are not met. As a consequence, data-driven semantic and geometric priors have been favored as regularizers, despite their bias toward training data and their neglect of broader movement dynamics. Departing from this practice, we propose a novel analytical approach that adapts the classical Lucas-Kanade method to dynamic Gaussian splatting. By leveraging the intrinsic properties of the forward warp field network, we derive an analytical velocity field that, through time integration, facilitates accurate scene flow computation. This enables the precise enforcement of motion constraints on warp fields, thus constraining both 2D motion and 3D positions of the Gaussians. Our method excels in reconstructing highly dynamic scenes with minimal camera movement, as demonstrated through experiments on both synthetic and real-world scenes.