Self-Supervised Learning for Endoscopic Video Analysis

TL;DR

This paper demonstrates that self-supervised learning, specifically Masked Siamese Networks, can significantly improve endoscopic video analysis by reducing the need for annotated data and achieving state-of-the-art results.

Contribution

The study applies SSL to endoscopy, creating large unlabeled datasets and showing that it enhances performance with less annotated data, a novel approach in this domain.

Findings

State-of-the-art performance in surgical phase recognition and polyp characterization

50% reduction in annotated data needed without performance loss

SSL effectively leverages unlabeled endoscopic videos for medical analysis

Abstract

Self-supervised learning (SSL) has led to important breakthroughs in computer vision by allowing learning from large amounts of unlabeled data. As such, it might have a pivotal role to play in biomedicine where annotating data requires a highly specialized expertise. Yet, there are many healthcare domains for which SSL has not been extensively explored. One such domain is endoscopy, minimally invasive procedures which are commonly used to detect and treat infections, chronic inflammatory diseases or cancer. In this work, we study the use of a leading SSL framework, namely Masked Siamese Networks (MSNs), for endoscopic video analysis such as colonoscopy and laparoscopy. To fully exploit the power of SSL, we create sizable unlabeled endoscopic video datasets for training MSNs. These strong image representations serve as a foundation for secondary training with limited annotated datasets, resulting in state-of-the-art performance in endoscopic benchmarks like surgical phase recognition during laparoscopy and colonoscopic polyp characterization. Additionally, we achieve a 50% reduction in annotated data size without sacrificing performance. Thus, our work provides evidence that SSL can dramatically reduce the need of annotated data in endoscopy.