AMNCutter: Affinity-Attention-Guided Multi-View Normalized Cutter for Unsupervised Surgical Instrument Segmentation

TL;DR

AMNCutter introduces a novel label-free unsupervised surgical instrument segmentation method that leverages graph-cutting loss and affinity prioritization, achieving state-of-the-art results and strong generalization in complex endoscopic scenarios.

Contribution

The paper presents a new unsupervised segmentation model using a graph-cutting loss and affinity-based supervision, eliminating the need for pseudo-labels and improving robustness.

Findings

Achieves state-of-the-art performance on multiple datasets.

Demonstrates superior robustness and generalization.

Provides a pre-trained model for surgical instrument segmentation.

Abstract

Surgical instrument segmentation (SIS) is pivotal for robotic-assisted minimally invasive surgery, assisting surgeons by identifying surgical instruments in endoscopic video frames. Recent unsupervised surgical instrument segmentation (USIS) methods primarily rely on pseudo-labels derived from low-level features such as color and optical flow, but these methods show limited effectiveness and generalizability in complex and unseen endoscopic scenarios. In this work, we propose a label-free unsupervised model featuring a novel module named Multi-View Normalized Cutter (m-NCutter). Different from previous USIS works, our model is trained using a graph-cutting loss function that leverages patch affinities for supervision, eliminating the need for pseudo-labels. The framework adaptively determines which affinities from which levels should be prioritized. Therefore, the low- and high-level features and their affinities are effectively integrated to train a label-free unsupervised model, showing superior effectiveness and generalization ability. We conduct comprehensive experiments across multiple SIS datasets to validate our approach's state-of-the-art (SOTA) performance, robustness, and exceptional potential as a pre-trained model. Our code is released at https://github.com/MingyuShengSMY/AMNCutter.