Mango-GS: Enhancing Spatio-Temporal Consistency in Dynamic Scenes Reconstruction using Multi-Frame Node-Guided 4D Gaussian Splatting

TL;DR

Mango-GS introduces a multi-frame, node-guided 4D Gaussian splatting framework that models motion dependencies with a temporal Transformer, achieving high-fidelity, temporally consistent dynamic scene reconstructions in real-time.

Contribution

The paper proposes a novel multi-frame, node-guided approach with a temporal Transformer for stable, high-quality dynamic scene reconstruction, addressing overfitting and correspondence drift issues.

Findings

Achieves state-of-the-art reconstruction quality.

Enables real-time rendering of dynamic scenes.

Provides robust motion modeling with sparse control nodes.

Abstract

Reconstructing dynamic 3D scenes with photorealistic detail and strong temporal coherence remains a significant challenge. Existing Gaussian splatting approaches for dynamic scene modeling often rely on per-frame optimization, which can overfit to instantaneous states instead of capturing underlying motion dynamics. To address this, we present Mango-GS, a multi-frame, node-guided framework for high-fidelity 4D reconstruction. Mango-GS leverages a temporal Transformer to model motion dependencies within a short window of frames, producing temporally consistent deformations. For efficiency, temporal modeling is confined to a sparse set of control nodes. Each node is represented by a decoupled canonical position and a latent code, providing a stable semantic anchor for motion propagation and preventing correspondence drift under large motion. Our framework is trained end-to-end, enhanced by an input masking strategy and two multi-frame losses to improve robustness. Extensive experiments demonstrate that Mango-GS achieves state-of-the-art reconstruction quality and real-time rendering speed, enabling high-fidelity reconstruction and interactive rendering of dynamic scenes.