Thoracic Surgery Video Analysis for Surgical Phase Recognition

TL;DR

This paper explores machine learning methods for recognizing surgical phases in thoracic surgery videos, demonstrating that video-based classifiers outperform image-based ones in accuracy, which can enhance surgical workflow analysis and training.

Contribution

It introduces and evaluates video-based classification models, particularly VideoMAE with Masked Video Distillation, for surgical phase recognition, showing significant accuracy improvements over image-based models.

Findings

VideoMAE with MVD achieves 72.9% accuracy.

Video classifiers outperform image classifiers in this task.

The approach aids surgical workflow understanding and training.

Abstract

This paper presents an approach for surgical phase recognition using video data, aiming to provide a comprehensive understanding of surgical procedures for automated workflow analysis. The advent of robotic surgery, digitized operating rooms, and the generation of vast amounts of data have opened doors for the application of machine learning and computer vision in the analysis of surgical videos. Among these advancements, Surgical Phase Recognition(SPR) stands out as an emerging technology that has the potential to recognize and assess the ongoing surgical scenario, summarize the surgery, evaluate surgical skills, offer surgical decision support, and facilitate medical training. In this paper, we analyse and evaluate both frame-based and video clipping-based phase recognition on thoracic surgery dataset consisting of 11 classes of phases. Specifically, we utilize ImageNet ViT for image-based classification and VideoMAE as the baseline model for video-based classification. We show that Masked Video Distillation(MVD) exhibits superior performance, achieving a top-1 accuracy of 72.9%, compared to 52.31% achieved by ImageNet ViT. These findings underscore the efficacy of video-based classifiers over their image-based counterparts in surgical phase recognition tasks.