Dy3DGS-SLAM: Monocular 3D Gaussian Splatting SLAM for Dynamic Environments

TL;DR

Dy3DGS-SLAM introduces a monocular RGB-based 3D Gaussian Splatting SLAM system capable of effectively tracking and reconstructing dynamic scenes by fusing optical flow and depth masks, achieving state-of-the-art results.

Contribution

This work is the first to develop a monocular 3D Gaussian Splatting SLAM method for dynamic environments, integrating dynamic mask fusion and novel loss functions.

Findings

Achieves state-of-the-art tracking in dynamic scenes

Outperforms existing RGB-D SLAM methods in dynamic environments

Effectively reconstructs dynamic scenes with monocular input

Abstract

Current Simultaneous Localization and Mapping (SLAM) methods based on Neural Radiance Fields (NeRF) or 3D Gaussian Splatting excel in reconstructing static 3D scenes but struggle with tracking and reconstruction in dynamic environments, such as real-world scenes with moving elements. Existing NeRF-based SLAM approaches addressing dynamic challenges typically rely on RGB-D inputs, with few methods accommodating pure RGB input. To overcome these limitations, we propose Dy3DGS-SLAM, the first 3D Gaussian Splatting (3DGS) SLAM method for dynamic scenes using monocular RGB input. To address dynamic interference, we fuse optical flow masks and depth masks through a probabilistic model to obtain a fused dynamic mask. With only a single network iteration, this can constrain tracking scales and refine rendered geometry. Based on the fused dynamic mask, we designed a novel motion loss to constrain the pose estimation network for tracking. In mapping, we use the rendering loss of dynamic pixels, color, and depth to eliminate transient interference and occlusion caused by dynamic objects. Experimental results demonstrate that Dy3DGS-SLAM achieves state-of-the-art tracking and rendering in dynamic environments, outperforming or matching existing RGB-D methods.