Active-Perceptive Motion Generation for Mobile Manipulation

TL;DR

This paper presents ActPerMoMa, an active perception approach for mobile manipulators that plans informative paths to improve grasping in cluttered, unknown environments, combining scene reconstruction and task success metrics.

Contribution

It introduces a receding horizon path planning method that balances information gain and grasp success, demonstrating effectiveness in simulation and real-world transfer.

Findings

Effective in simulated cluttered scenes

Improves grasp success rate

Transfers well to real-world scenarios

Abstract

Mobile Manipulation (MoMa) systems incorporate the benefits of mobility and dexterity, due to the enlarged space in which they can move and interact with their environment. However, even when equipped with onboard sensors, e.g., an embodied camera, extracting task-relevant visual information in unstructured and cluttered environments, such as households, remains challenging. In this work, we introduce an active perception pipeline for mobile manipulators to generate motions that are informative toward manipulation tasks, such as grasping in unknown, cluttered scenes. Our proposed approach, ActPerMoMa, generates robot paths in a receding horizon fashion by sampling paths and computing path-wise utilities. These utilities trade-off maximizing the visual Information Gain (IG) for scene reconstruction and the task-oriented objective, e.g., grasp success, by maximizing grasp reachability. We show the efficacy of our method in simulated experiments with a dual-arm TIAGo++ MoMa robot performing mobile grasping in cluttered scenes with obstacles. We empirically analyze the contribution of various utilities and parameters, and compare against representative baselines both with and without active perception objectives. Finally, we demonstrate the transfer of our mobile grasping strategy to the real world, indicating a promising direction for active-perceptive MoMa.