Learning Autonomous Exploration and Mapping with Semantic Vision

TL;DR

This paper introduces a novel end-to-end learning framework for autonomous exploration and mapping using visual inputs, integrating semantic interpretation, geometry, and decision-making in a differentiable pipeline.

Contribution

It presents the first learning-based approach that combines semantic segmentation, geometry, and exploration in a unified, end-to-end trainable system for robotic mapping.

Findings

Demonstrates improved exploration efficiency over baseline methods

Validates approach in simulated real-world environments

Achieves integrated semantic mapping and exploration decision-making

Abstract

We address the problem of autonomous exploration and mapping for a mobile robot using visual inputs. Exploration and mapping is a well-known and key problem in robotics, the goal of which is to enable a robot to explore a new environment autonomously and create a map for future usage. Different to classical methods, we propose a learning-based approach this work based on semantic interpretation of visual scenes. Our method is based on a deep network consisting of three modules: semantic segmentation network, mapping using camera geometry and exploration action network. All modules are differentiable, so the whole pipeline is trained end-to-end based on actor-critic framework. Our network makes action decision step by step and generates the free space map simultaneously. To our best knowledge, this is the first algorithm that formulate exploration and mapping into learning framework. We validate our approach in simulated real world environments and demonstrate performance gains over competitive baseline approaches.