Open-Source Multi-Viewpoint Surgical Telerobotics

TL;DR

This paper introduces an open-source multi-viewpoint surgical telerobotics system that enhances visualization, collaboration, and perception in minimally invasive surgery through synchronized multi-view 3D measurements and control.

Contribution

It presents a novel multi-viewpoint, multi-sensor robotic surgery system with open-source implementation, enabling improved visualization, shared autonomy, and advanced scene understanding in MIS.

Findings

System successfully integrates multi-view sensors and control logic.

Enables independent viewpoint control for surgeons.

Facilitates real-time 3D perception and potential algorithmic assistance.

Abstract

As robots for minimally invasive surgery (MIS) gradually become more accessible and modular, we believe there is a great opportunity to rethink and expand the visualization and control paradigms that have characterized surgical teleoperation since its inception. We conjecture that introducing one or more additional adjustable viewpoints in the abdominal cavity would not only unlock novel visualization and collaboration strategies for surgeons but also substantially boost the robustness of machine perception toward shared autonomy. Immediate advantages include controlling a second viewpoint and teleoperating surgical tools from a different perspective, which would allow collaborating surgeons to adjust their views independently and still maneuver their robotic instruments intuitively. Furthermore, we believe that capturing synchronized multi-view 3D measurements of the patient's anatomy would unlock advanced scene representations. Accurate real-time intraoperative 3D perception will allow algorithmic assistants to directly control one or more robotic instruments and/or robotic cameras. Toward these goals, we are building a synchronized multi-viewpoint, multi-sensor robotic surgery system by integrating high-performance vision components and upgrading the da Vinci Research Kit control logic. This short paper reports a functional summary of our setup and elaborates on its potential impacts in research and future clinical practice. By fully open-sourcing our system, we will enable the research community to reproduce our setup, improve it, and develop powerful algorithms, effectively boosting clinical translation of cutting-edge research.