SuPer: A Surgical Perception Framework for Endoscopic Tissue Manipulation with Surgical Robotics

TL;DR

SuPer is a novel surgical perception framework that enables real-time 3D mapping and instrument tracking in endoscopic surgery, facilitating autonomous soft tissue manipulation with high accuracy.

Contribution

This work introduces SuPer, a new perception framework combining model-based and model-free tracking for deformable environments in robotic surgery.

Findings

Successfully completed soft tissue manipulation tasks

Real-time implementation on da Vinci Surgical System

High accuracy in surgical environment mapping

Abstract

Traditional control and task automation have been successfully demonstrated in a variety of structured, controlled environments through the use of highly specialized modeled robotic systems in conjunction with multiple sensors. However, the application of autonomy in endoscopic surgery is very challenging, particularly in soft tissue work, due to the lack of high-quality images and the unpredictable, constantly deforming environment. In this work, we propose a novel surgical perception framework, SuPer, for surgical robotic control. This framework continuously collects 3D geometric information that allows for mapping a deformable surgical field while tracking rigid instruments within the field. To achieve this, a model-based tracker is employed to localize the surgical tool with a kinematic prior in conjunction with a model-free tracker to reconstruct the deformable environment and provide an estimated point cloud as a mapping of the environment. The proposed framework was implemented on the da Vinci Surgical System in real-time with an end-effector controller where the target configurations are set and regulated through the framework. Our proposed framework successfully completed soft tissue manipulation tasks with high accuracy. The demonstration of this novel framework is promising for the future of surgical autonomy. In addition, we provide our dataset for further surgical research.