FastGrasp: Learning-based Whole-body Control method for Fast Dexterous Grasping with Mobile Manipulators

TL;DR

FastGrasp is a learning-based framework enabling mobile robots to perform fast, robust, and dexterous grasping by integrating grasp guidance, whole-body control, and tactile feedback, validated in simulation and real-world tests.

Contribution

The paper introduces a novel two-stage reinforcement learning approach combined with tactile sensing for real-time, whole-body mobile grasping, addressing impact stabilization and generalization challenges.

Findings

Achieves robust grasping across diverse objects in simulation and real-world.

Demonstrates effective sim-to-real transfer for mobile grasping tasks.

Outperforms existing methods in speed and stability of grasping.

Abstract

Fast grasping is critical for mobile robots in logistics, manufacturing, and service applications. Existing methods face fundamental challenges in impact stabilization under high-speed motion, real-time whole-body coordination, and generalization across diverse objects and scenarios, limited by fixed bases, simple grippers, or slow tactile response capabilities. We propose \textbf{FastGrasp}, a learning-based framework that integrates grasp guidance, whole-body control, and tactile feedback for mobile fast grasping. Our two-stage reinforcement learning strategy first generates diverse grasp candidates via conditional variational autoencoder conditioned on object point clouds, then executes coordinated movements of mobile base, arm, and hand guided by optimal grasp selection. Tactile sensing enables real-time grasp adjustments to handle impact effects and object variations. Extensive experiments demonstrate superior grasping performance in both simulation and real-world scenarios, achieving robust manipulation across diverse object geometries through effective sim-to-real transfer.