"Train one, Classify one, Teach one" -- Cross-surgery transfer learning for surgical step recognition

TL;DR

This paper introduces TSAN, a transfer learning architecture for surgical step recognition across multiple laparoscopic procedures, achieving high accuracy with limited labeled data by leveraging self-supervised pre-training.

Contribution

The study proposes TSAN, a novel architecture optimized for transfer learning in surgical workflow recognition, and demonstrates its effectiveness across four different laparoscopic surgeries.

Findings

TSAN outperforms other architectures in transfer learning scenarios.

Pre-training with self-supervised learning improves recognition accuracy.

Achieves over 90% accuracy transferring from Cholecystectomy to other procedures.

Abstract

Prior work demonstrated the ability of machine learning to automatically recognize surgical workflow steps from videos. However, these studies focused on only a single type of procedure. In this work, we analyze, for the first time, surgical step recognition on four different laparoscopic surgeries: Cholecystectomy, Right Hemicolectomy, Sleeve Gastrectomy, and Appendectomy. Inspired by the traditional apprenticeship model, in which surgical training is based on the Halstedian method, we paraphrase the "see one, do one, teach one" approach for the surgical intelligence domain as "train one, classify one, teach one". In machine learning, this approach is often referred to as transfer learning. To analyze the impact of transfer learning across different laparoscopic procedures, we explore various time-series architectures and examine their performance on each target domain. We introduce a new architecture, the Time-Series Adaptation Network (TSAN), an architecture optimized for transfer learning of surgical step recognition, and we show how TSAN can be pre-trained using self-supervised learning on a Sequence Sorting task. Such pre-training enables TSAN to learn workflow steps of a new laparoscopic procedure type from only a small number of labeled samples from the target procedure. Our proposed architecture leads to better performance compared to other possible architectures, reaching over 90% accuracy when transferring from laparoscopic Cholecystectomy to the other three procedure types.