Finding Things in the Unknown: Semantic Object-Centric Exploration with an MAV

TL;DR

This paper presents a semantic, object-centric exploration framework for autonomous MAVs that enhances environment mapping with object-level understanding, enabling efficient target object detection and reconstruction in unknown spaces.

Contribution

The work introduces a novel exploration approach that combines semantic object detection with environment mapping, including a new objective function and a MAV simulation platform.

Findings

Efficiently finds specific objects in unknown environments.

Reconstructs objects with high accuracy using adaptive distance constraints.

Demonstrates real-world deployment on a drone with an RGB-D camera.

Abstract

Exploration of unknown space with an autonomous mobile robot is a well-studied problem. In this work we broaden the scope of exploration, moving beyond the pure geometric goal of uncovering as much free space as possible. We believe that for many practical applications, exploration should be contextualised with semantic and object-level understanding of the environment for task-specific exploration. Here, we study the task of both finding specific objects in unknown space as well as reconstructing them to a target level of detail. We therefore extend our environment reconstruction to not only consist of a background map, but also object-level and semantically fused submaps. Importantly, we adapt our previous objective function of uncovering as much free space as possible in as little time as possible with two additional elements: first, we require a maximum observation distance of background surfaces to ensure target objects are not missed by image-based detectors because they are too small to be detected. Second, we require an even smaller maximum distance to the found objects in order to reconstruct them with the desired accuracy. We further created a Micro Aerial Vehicle (MAV) semantic exploration simulator based on Habitat in order to quantitatively demonstrate how our framework can be used to efficiently find specific objects as part of exploration. Finally, we showcase this capability can be deployed in real-world scenes involving our drone equipped with an Intel RealSense D455 RGB-D camera.