Efficient Manipulation-Enhanced Semantic Mapping With Uncertainty-Informed Action Selection

TL;DR

This paper introduces a manipulation-enhanced semantic mapping framework that uses uncertainty estimates to guide active sensing and targeted object manipulation, significantly improving mapping accuracy and efficiency in cluttered environments.

Contribution

The work integrates evidential mapping with reinforcement learning for next-best view planning and proposes an uncertainty-informed push strategy for occlusion reduction.

Findings

Achieves 95% reduction in planning time compared to state-of-the-art methods.

Reduces object displacement during mapping.

Enables accurate mapping of cluttered scenes in real-world scenarios.

Abstract

Service robots operating in cluttered human environments such as homes, offices, and schools cannot rely on predefined object arrangements and must continuously update their semantic and spatial estimates while dealing with possible frequent rearrangements. Efficient and accurate mapping under such conditions demands selecting informative viewpoints and targeted manipulations to reduce occlusions and uncertainty. In this work, we present a manipulation-enhanced semantic mapping framework for occlusion-heavy shelf scenes that integrates evidential metric-semantic mapping with reinforcement-learning-based next-best view planning and targeted action selection. Our method thereby exploits uncertainty estimates from Dirichlet and Beta distributions in the map prediction networks to guide both active sensor placement and object manipulation, focusing on areas with high uncertainty and selecting actions with high expected information gain. Furthermore, we introduce an uncertainty-informed push strategy that targets occlusion-critical objects and generates minimally invasive actions to reveal hidden regions by reducing overall uncertainty in the scene. The experimental evaluation shows that our framework enables to accurately map cluttered scenes, while substantially reducing object displacement and achieving a 95% reduction in planning time compared to the state-of-the-art, thereby realizing real-world applicability.