Runtime Detection of Executional Errors in Robot-Assisted Surgery

TL;DR

This paper introduces a real-time error detection system for robot-assisted surgery using Siamese neural networks, which effectively distinguish normal from erroneous surgical trajectories with high accuracy, enhancing safety in minimally-invasive procedures.

Contribution

The study develops a novel Siamese neural network-based system that leverages surgical context to detect executional errors in real-time, outperforming traditional CNN and LSTM models.

Findings

Siamese networks achieve micro F1 scores of 0.94 and 0.95.

Siamese networks outperform CNN and LSTM in error detection accuracy.

Gesture-specific Siamese networks perform better than gesture-nonspecific models.

Abstract

Despite significant developments in the design of surgical robots and automated techniques for objective evaluation of surgical skills, there are still challenges in ensuring safety in robot-assisted minimally-invasive surgery (RMIS). This paper presents a runtime monitoring system for the detection of executional errors during surgical tasks through the analysis of kinematic data. The proposed system incorporates dual Siamese neural networks and knowledge of surgical context, including surgical tasks and gestures, their distributional similarities, and common error modes, to learn the differences between normal and erroneous surgical trajectories from small training datasets. We evaluate the performance of the error detection using Siamese networks compared to single CNN and LSTM networks trained with different levels of contextual knowledge and training data, using the dry-lab demonstrations of the Suturing and Needle Passing tasks from the JIGSAWS dataset. Our results show that gesture specific task nonspecific Siamese networks obtain micro F1 scores of 0.94 (Siamese-CNN) and 0.95 (Siamese-LSTM), and perform better than single CNN (0.86) and LSTM (0.87) networks. These Siamese networks also outperform gesture nonspecific task specific Siamese-CNN and Siamese-LSTM models for Suturing and Needle Passing.