DynamicGSG: Dynamic 3D Gaussian Scene Graphs for Environment Adaptation

TL;DR

DynamicGSG introduces a novel system for real-time, high-fidelity scene graph construction that dynamically updates to adapt to environmental changes, enhancing robot perception and long-term task performance.

Contribution

It presents a dynamic, hierarchical scene graph system leveraging Gaussian splatting and vision language models for environment understanding and adaptation.

Findings

Effective in semantic segmentation and object retrieval

Demonstrates real-time environment adaptation in laboratory settings

Improves reconstruction quality through Gaussian scene graph updates

Abstract

In real-world scenarios, environment changes caused by human or agent activities make it extremely challenging for robots to perform various long-term tasks. Recent works typically struggle to effectively understand and adapt to dynamic environments due to the inability to update their environment representations in memory according to environment changes and lack of fine-grained reconstruction of the environments. To address these challenges, we propose DynamicGSG, a dynamic, high-fidelity, open-vocabulary scene graph construction system leveraging Gaussian splatting. DynamicGSG builds hierarchical scene graphs using advanced vision language models to represent the spatial and semantic relationships between objects in the environments, utilizes a joint feature loss we designed to supervise Gaussian instance grouping while optimizing the Gaussian maps, and locally updates the Gaussian scene graphs according to real environment changes for long-term environment adaptation. Experiments and ablation studies demonstrate the performance and efficacy of our proposed method in terms of semantic segmentation, language-guided object retrieval, and reconstruction quality. Furthermore, we validate the dynamic updating capabilities of our system in real laboratory environments. The source code and supplementary experimental materials will be released at:~\href{https://github.com/GeLuzhou/Dynamic-GSG}{https://github.com/GeLuzhou/Dynamic-GSG}.