COMBO-Grasp: Learning Constraint-Based Manipulation for Bimanual Occluded Grasping

TL;DR

COMBO-Grasp introduces a bimanual manipulation approach that combines a constraint policy and a reinforcement learning grasping policy, guided by a value function, to improve occluded object grasping in complex environments.

Contribution

It presents a novel learning-based framework that integrates constraint-based stabilization with RL-driven grasping, enhancing bimanual coordination and generalization to unseen objects.

Findings

Significantly outperforms baseline methods in success rates

Successfully generalizes to unseen objects in simulation and real-world

Improves task success through value-guided policy coordination

Abstract

This paper addresses the challenge of occluded robot grasping, i.e. grasping in situations where the desired grasp poses are kinematically infeasible due to environmental constraints such as surface collisions. Traditional robot manipulation approaches struggle with the complexity of non-prehensile or bimanual strategies commonly used by humans in these circumstances. State-of-the-art reinforcement learning (RL) methods are unsuitable due to the inherent complexity of the task. In contrast, learning from demonstration requires collecting a significant number of expert demonstrations, which is often infeasible. Instead, inspired by human bimanual manipulation strategies, where two hands coordinate to stabilise and reorient objects, we focus on a bimanual robotic setup to tackle this challenge. In particular, we introduce Constraint-based Manipulation for Bimanual Occluded Grasping (COMBO-Grasp), a learning-based approach which leverages two coordinated policies: a constraint policy trained using self-supervised datasets to generate stabilising poses and a grasping policy trained using RL that reorients and grasps the target object. A key contribution lies in value function-guided policy coordination. Specifically, during RL training for the grasping policy, the constraint policy's output is refined through gradients from a jointly trained value function, improving bimanual coordination and task performance. Lastly, COMBO-Grasp employs teacher-student policy distillation to effectively deploy point cloud-based policies in real-world environments. Empirical evaluations demonstrate that COMBO-Grasp significantly improves task success rates compared to competitive baseline approaches, with successful generalisation to unseen objects in both simulated and real-world environments.