Endo-SemiS: Towards Robust Semi-Supervised Image Segmentation for Endoscopic Video

TL;DR

Endo-SemiS introduces a semi-supervised framework for endoscopic video segmentation that leverages multiple strategies including cross-supervision, uncertainty-guided pseudo-labels, and mutual learning to improve accuracy with limited labeled data.

Contribution

The paper proposes a novel semi-supervised segmentation method specifically designed for endoscopic videos, integrating multiple strategies and a spatiotemporal correction network to enhance performance.

Findings

Outperforms state-of-the-art methods on clinical endoscopy datasets

Effective with limited labeled data

Improves segmentation accuracy in endoscopic videos

Abstract

In this paper, we present Endo-SemiS, a semi-supervised segmentation framework for providing reliable segmentation of endoscopic video frames with limited annotation. EndoSemiS uses 4 strategies to improve performance by effectively utilizing all available data, particularly unlabeled data: (1) Cross-supervision between two individual networks that supervise each other; (2) Uncertainty-guided pseudo-labels from unlabeled data, which are generated by selecting high-confidence regions to improve their quality; (3) Joint pseudolabel supervision, which aggregates reliable pixels from the pseudo-labels of both networks to provide accurate supervision for unlabeled data; and (4) Mutual learning, where both networks learn from each other at the feature and image levels, reducing variance and guiding them toward a consistent solution. Additionally, a separate corrective network that utilizes spatiotemporal information from endoscopy video to improve segmentation performance. Endo-SemiS is evaluated on two clinical applications: kidney stone laser lithotomy from ureteroscopy and polyp screening from colonoscopy. Compared to state-of-the-art segmentation methods, Endo-SemiS substantially achieves superior results on both datasets with limited labeled data. The code is publicly available at https://github.com/MedICL-VU/Endo-SemiS