Surgical Activity Recognition in Robot-Assisted Radical Prostatectomy using Deep Learning

TL;DR

This paper introduces deep learning methods, including a modified InceptionV3 model called RP-Net, to automatically recognize surgical tasks in robot-assisted prostatectomy, enabling more precise surgical quality metrics.

Contribution

The study develops and evaluates RP-Net, a novel deep learning model for surgical activity recognition, demonstrating improved accuracy over existing models in robot-assisted prostatectomy.

Findings

RP-Net achieves 80.9% precision and 76.7% recall.

Automatic activity recognition is feasible during RARP.

Deep learning models outperform RNN and CNN baselines.

Abstract

Adverse surgical outcomes are costly to patients and hospitals. Approaches to benchmark surgical care are often limited to gross measures across the entire procedure despite the performance of particular tasks being largely responsible for undesirable outcomes. In order to produce metrics from tasks as opposed to the whole procedure, methods to recognize automatically individual surgical tasks are needed. In this paper, we propose several approaches to recognize surgical activities in robot-assisted minimally invasive surgery using deep learning. We collected a clinical dataset of 100 robot-assisted radical prostatectomies (RARP) with 12 tasks each and propose `RP-Net', a modified version of InceptionV3 model, for image based surgical activity recognition. We achieve an average precision of 80.9% and average recall of 76.7% across all tasks using RP-Net which out-performs all other RNN and CNN based models explored in this paper. Our results suggest that automatic surgical activity recognition during RARP is feasible and can be the foundation for advanced analytics.