Dynamic Objects Segmentation for Visual Localization in Urban Environments

TL;DR

This paper introduces a semi-supervised convolutional neural network approach to detect dynamic objects in urban environments, enhancing the robustness of visual localization and mapping for mobile robots.

Contribution

The work presents a novel semi-supervised training method combining synthetic and real data for dynamic object detection in urban scenes, improving localization robustness.

Findings

Reliable detection of various dynamic objects in urban environments.

Effective generalization demonstrated on multiple datasets.

Enhanced robustness of visual localization in crowded scenes.

Abstract

Visual localization and mapping is a crucial capability to address many challenges in mobile robotics. It constitutes a robust, accurate and cost-effective approach for local and global pose estimation within prior maps. Yet, in highly dynamic environments, like crowded city streets, problems arise as major parts of the image can be covered by dynamic objects. Consequently, visual odometry pipelines often diverge and the localization systems malfunction as detected features are not consistent with the precomputed 3D model. In this work, we present an approach to automatically detect dynamic object instances to improve the robustness of vision-based localization and mapping in crowded environments. By training a convolutional neural network model with a combination of synthetic and real-world data, dynamic object instance masks are learned in a semi-supervised way. The real-world data can be collected with a standard camera and requires minimal further post-processing. Our experiments show that a wide range of dynamic objects can be reliably detected using the presented method. Promising performance is demonstrated on our own and also publicly available datasets, which also shows the generalization capabilities of this approach.