Learning to Localize, Grasp, and Hand Over Unmodified Surgical Needles

TL;DR

This paper introduces HOUSTON, a novel algorithm enabling robotic surgical assistants to localize, grasp, and hand over unmodified surgical needles using active sensing and multi-camera systems, achieving high success rates without modifying the needles.

Contribution

It presents the first method for handover of unmodified surgical needles, combining learned active sensing with multi-camera high-precision grasping in robotic surgery.

Findings

Success rate of 96.7% in physical experiments

Average of 32.4 handovers before failure

Achieves 75-92.9% success on unseen needles

Abstract

Robotic Surgical Assistants (RSAs) are commonly used to perform minimally invasive surgeries by expert surgeons. However, long procedures filled with tedious and repetitive tasks such as suturing can lead to surgeon fatigue, motivating the automation of suturing. As visual tracking of a thin reflective needle is extremely challenging, prior work has modified the needle with nonreflective contrasting paint. As a step towards automation of a suturing subtask without modifying the needle, we propose HOUSTON: Handoff of Unmodified, Surgical, Tool-Obstructed Needles, a problem and algorithm that uses a learned active sensing policy with a stereo camera to localize and align the needle into a visible and accessible pose for the other arm. To compensate for robot positioning and needle perception errors, the algorithm then executes a high-precision grasping motion that uses multiple cameras. In physical experiments using the da Vinci Research Kit (dVRK), HOUSTON successfully passes unmodified surgical needles with a success rate of 96.7% and is able to perform handover sequentially between the arms 32.4 times on average before failure. On needles unseen in training, HOUSTON achieves a success rate of 75 - 92.9%. To our knowledge, this work is the first to study handover of unmodified surgical needles. See https://tinyurl.com/houston-surgery for additional materials.