HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis

TL;DR

HoliGS introduces a deformable Gaussian splatting framework that efficiently reconstructs and renders large-scale dynamic scenes from monocular videos, outperforming prior methods in quality and speed.

Contribution

It presents a hierarchical deformation approach using invertible neural flows for scalable, accurate embodied view synthesis from monocular videos.

Findings

Achieves superior reconstruction quality on challenging datasets.

Reduces training and rendering time significantly.

Effectively handles large-scale dynamic environments with multiple actors.

Abstract

We propose HoliGS, a novel deformable Gaussian splatting framework that addresses embodied view synthesis from long monocular RGB videos. Unlike prior 4D Gaussian splatting and dynamic NeRF pipelines, which struggle with training overhead in minute-long captures, our method leverages invertible Gaussian Splatting deformation networks to reconstruct large-scale, dynamic environments accurately. Specifically, we decompose each scene into a static background plus time-varying objects, each represented by learned Gaussian primitives undergoing global rigid transformations, skeleton-driven articulation, and subtle non-rigid deformations via an invertible neural flow. This hierarchical warping strategy enables robust free-viewpoint novel-view rendering from various embodied camera trajectories by attaching Gaussians to a complete canonical foreground shape (\eg, egocentric or third-person follow), which may involve substantial viewpoint changes and interactions between multiple actors. Our experiments demonstrate that \ourmethod~ achieves superior reconstruction quality on challenging datasets while significantly reducing both training and rendering time compared to state-of-the-art monocular deformable NeRFs. These results highlight a practical and scalable solution for EVS in real-world scenarios. The source code will be released.