Bimanual Regrasping for Suture Needles using Reinforcement Learning for Rapid Motion Planning

TL;DR

This paper introduces a reinforcement learning-based method for rapid bimanual regrasping of suture needles, achieving high success rates and fast planning times, adaptable to different robot configurations and needle poses.

Contribution

The work presents a novel RL approach with ego-centric state/action spaces for efficient, adaptable bimanual needle regrasping, incorporating demonstrations to enhance learning speed.

Findings

97% success rate in simulation for single pass

Real-world success rate of 73.3% with RGB-based pose estimation

Planning time under 0.085 seconds in both simulation and real-world

Abstract

Regrasping a suture needle is an important yet time-consuming process in suturing. To bring efficiency into regrasping, prior work either designs a task-specific mechanism or guides the gripper toward some specific pick-up point for proper grasping of a needle. Yet, these methods are usually not deployable when the working space is changed. Therefore, in this work, we present rapid trajectory generation for bimanual needle regrasping via reinforcement learning (RL). Demonstrations from a sampling-based motion planning algorithm is incorporated to speed up the learning. In addition, we propose the ego-centric state and action spaces for this bimanual planning problem, where the reference frames are on the end-effectors instead of some fixed frame. Thus, the learned policy can be directly applied to any feasible robot configuration. Our experiments in simulation show that the success rate of a single pass is 97%, and the planning time is 0.0212s on average, which outperforms other widely used motion planning algorithms. For the real-world experiments, the success rate is 73.3% if the needle pose is reconstructed from an RGB image, with a planning time of 0.0846s and a run time of 5.1454s. If the needle pose is known beforehand, the success rate becomes 90.5%, with a planning time of 0.0807s and a run time of 2.8801s.