SUPR-GAN: SUrgical PRediction GAN for Event Anticipation in Laparoscopic and Robotic Surgery

TL;DR

This paper introduces SUPR-GAN, a novel generative adversarial network designed to predict future surgical phases in laparoscopic and robotic surgeries, enhancing intraoperative decision-making and workflow understanding.

Contribution

It presents a new GAN-based model for predicting surgical workflow transitions, outperforming existing methods and providing a framework for future intraoperative AI applications.

Findings

SUPR-GAN accurately predicts future surgical phases.

The model effectively captures phase transition dynamics.

Surgeons qualitatively favor the predictions made by SUPR-GAN.

Abstract

Comprehension of surgical workflow is the foundation upon which artificial intelligence (AI) and machine learning (ML) holds the potential to assist intraoperative decision-making and risk mitigation. In this work, we move beyond mere identification of past surgical phases, into the prediction of future surgical steps and specification of the transitions between them. We use a novel Generative Adversarial Network (GAN) formulation to sample future surgical phases trajectories conditioned on past video frames from laparoscopic cholecystectomy (LC) videos and compare it to state-of-the-art approaches for surgical video analysis and alternative prediction methods. We demonstrate the GAN formulation's effectiveness through inferring and predicting the progress of LC videos. We quantify the horizon-accuracy trade-off and explored average performance, as well as the performance on the more challenging, and clinically relevant transitions between phases. Furthermore, we conduct a survey, asking 16 surgeons of different specialties and educational levels to qualitatively evaluate predicted surgery phases.