SR-Mamba: Effective Surgical Phase Recognition with State Space Model

TL;DR

SR-Mamba is a novel attention-free model that effectively captures long-range temporal dependencies in surgical videos, achieving state-of-the-art phase recognition performance with simplified training.

Contribution

It introduces a bidirectional Mamba decoder tailored for surgical videos, enabling single-step training and improved accuracy over previous methods.

Findings

Achieves state-of-the-art results on Cholec80 and CATARACTS datasets.

Simplifies training process with single-step optimization.

Outperforms existing models in surgical phase recognition.

Abstract

Surgical phase recognition is crucial for enhancing the efficiency and safety of computer-assisted interventions. One of the fundamental challenges involves modeling the long-distance temporal relationships present in surgical videos. Inspired by the recent success of Mamba, a state space model with linear scalability in sequence length, this paper presents SR-Mamba, a novel attention-free model specifically tailored to meet the challenges of surgical phase recognition. In SR-Mamba, we leverage a bidirectional Mamba decoder to effectively model the temporal context in overlong sequences. Moreover, the efficient optimization of the proposed Mamba decoder facilitates single-step neural network training, eliminating the need for separate training steps as in previous works. This single-step training approach not only simplifies the training process but also ensures higher accuracy, even with a lighter spatial feature extractor. Our SR-Mamba establishes a new benchmark in surgical video analysis by demonstrating state-of-the-art performance on the Cholec80 and CATARACTS Challenge datasets. The code is accessible at https://github.com/rcao-hk/SR-Mamba.