Dynamic Gaussian Scene Reconstruction from Unsynchronized Videos

TL;DR

This paper introduces a novel temporal alignment method for 4D Gaussian Splatting that enables high-quality dynamic scene reconstruction from unsynchronized multi-view videos, addressing real-world temporal misalignment issues.

Contribution

It presents a coarse-to-fine alignment strategy that estimates and corrects camera time shifts, improving 4DGS reconstruction robustness with asynchronous videos.

Findings

Effective handling of temporally misaligned videos

Significant improvement over baseline reconstruction quality

Compatible with existing 4DGS frameworks

Abstract

Multi-view video reconstruction plays a vital role in computer vision, enabling applications in film production, virtual reality, and motion analysis. While recent advances such as 4D Gaussian Splatting (4DGS) have demonstrated impressive capabilities in dynamic scene reconstruction, they typically rely on the assumption that input video streams are temporally synchronized. However, in real-world scenarios, this assumption often fails due to factors like camera trigger delays or independent recording setups, leading to temporal misalignment across views and reduced reconstruction quality. To address this challenge, a novel temporal alignment strategy is proposed for high-quality 4DGS reconstruction from unsynchronized multi-view videos. Our method features a coarse-to-fine alignment module that estimates and compensates for each camera's time shift. The method first determines a coarse, frame-level offset and then refines it to achieve sub-frame accuracy. This strategy can be integrated as a readily integrable module into existing 4DGS frameworks, enhancing their robustness when handling asynchronous data. Experiments show that our approach effectively processes temporally misaligned videos and significantly enhances baseline methods.