# Robust Dense Mapping for Large-Scale Dynamic Environments

## TL;DR

This paper introduces a stereo-based dense mapping system for large-scale dynamic urban environments that separately reconstructs static background, moving objects, and potentially moving objects, enhancing robotic navigation and planning.

## Contribution

It presents a novel approach combining semantic segmentation and scene flow to distinguish and reconstruct static and dynamic elements separately in large-scale scenes.

## Key findings

- Capable of real-time operation at 2.5Hz on a PC
- Effectively models objects with potential to change from static to dynamic
- Improves scalability through map pruning techniques

## Abstract

We present a stereo-based dense mapping algorithm for large-scale dynamic urban environments. In contrast to other existing methods, we simultaneously reconstruct the static background, the moving objects, and the potentially moving but currently stationary objects separately, which is desirable for high-level mobile robotic tasks such as path planning in crowded environments. We use both instance-aware semantic segmentation and sparse scene flow to classify objects as either background, moving, or potentially moving, thereby ensuring that the system is able to model objects with the potential to transition from static to dynamic, such as parked cars. Given camera poses estimated from visual odometry, both the background and the (potentially) moving objects are reconstructed separately by fusing the depth maps computed from the stereo input. In addition to visual odometry, sparse scene flow is also used to estimate the 3D motions of the detected moving objects, in order to reconstruct them accurately. A map pruning technique is further developed to improve reconstruction accuracy and reduce memory consumption, leading to increased scalability. We evaluate our system thoroughly on the well-known KITTI dataset. Our system is capable of running on a PC at approximately 2.5Hz, with the primary bottleneck being the instance-aware semantic segmentation, which is a limitation we hope to address in future work. The source code is available from the project website (http://andreibarsan.github.io/dynslam).