Augmented Reality Predictive Displays to Help Mitigate the Effects of Delayed Telesurgery

TL;DR

This paper introduces SARPD, a stereoscopic AR predictive display designed for telesurgery that reduces task completion time under latency without increasing errors.

Contribution

It presents the first AR-based predictive display tailored for teleoperated surgical robots, addressing real-time 3D tracking challenges in surgical environments.

Findings

SARPD significantly reduces task completion time under delay.

SARPD does not increase error rates during telesurgery.

User study confirms effectiveness of SARPD in simulated surgical tasks.

Abstract

Surgical robots offer the exciting potential for remote telesurgery, but advances are needed to make this technology efficient and accurate to ensure patient safety. Achieving these goals is hindered by the deleterious effects of latency between the remote operator and the bedside robot. Predictive displays have found success in overcoming these effects by giving the operator immediate visual feedback. However, previously developed predictive displays can not be directly applied to telesurgery due to the unique challenges in tracking the 3D geometry of the surgical environment. In this paper, we present the first predictive display for teleoperated surgical robots. The predicted display is stereoscopic, utilizes Augmented Reality (AR) to show the predicted motions alongside the complex tissue found in-situ within surgical environments, and overcomes the challenges in accurately tracking slave-tools in real-time. We call this a Stereoscopic AR Predictive Display (SARPD). To test the SARPD's performance, we conducted a user study with ten participants on the da Vinci\textregistered{} Surgical System. The results showed with statistical significance that using SARPD decreased time to complete task while having no effect on error rates when operating under delay.