Visibility-Aware Mobile Grasping in Dynamic Environments

TL;DR

This paper presents a unified mobile grasping system that combines active perception and hierarchical planning to safely operate in dynamic, unknown environments with limited visibility.

Contribution

It introduces an integrated approach with low-level and high-level planning components that adaptively explore and navigate dynamic environments for mobile grasping.

Findings

Achieved 68.8% success in unknown static environments.

Achieved 58.0% success in unknown dynamic environments.

Significantly outperformed previous methods with over 20% higher success rates.

Abstract

This paper addresses the problem of mobile grasping in dynamic, unknown environments where a robot must operate under a limited field-of-view. The fundamental challenge is the inherent trade-off between ``seeing'' around to reduce environmental uncertainty and ``moving'' the body to achieve task progress in a high-dimensional configuration space, subject to visibility constraints. Previous approaches often assume known or static environments and decouple these objectives, failing to guarantee safety when unobserved dynamic obstacles intersect the robot's path during manipulation. In this paper, we propose a unified mobile grasping system comprising two core components: (1) an iterative low-level whole-body planner coupled with velocity-aware active perception to navigate dynamic environments safely; and (2) a hierarchical high-level planner based on behavior trees that adaptively generates subgoals to guide the robot through exploration and runtime failures. We provide experimental results across 400 randomized simulation scenarios and real-world deployment on a Fetch mobile manipulator. Results show that our system achieves a success rate of 68.8\% and 58.0\% in unknown static and dynamic environments, respectively, significantly boosting success rates by 22.8\% and 18.0\% over the \nam approach in both unknown static and dynamic environments, with improved collision safety.