ARST: Auto-Regressive Surgical Transformer for Phase Recognition from Laparoscopic Videos

TL;DR

This paper introduces ARST, an auto-regressive transformer model for real-time surgical phase recognition from laparoscopic videos, improving accuracy and consistency over existing methods.

Contribution

It is the first to incorporate auto-regression into transformer-based surgical phase recognition, enhancing phase correlation modeling and inference stability.

Findings

Outperforms state-of-the-art methods in accuracy.

Achieves 66 fps inference rate.

Demonstrates improved phase consistency.

Abstract

Phase recognition plays an essential role for surgical workflow analysis in computer assisted intervention. Transformer, originally proposed for sequential data modeling in natural language processing, has been successfully applied to surgical phase recognition. Existing works based on transformer mainly focus on modeling attention dependency, without introducing auto-regression. In this work, an Auto-Regressive Surgical Transformer, referred as ARST, is first proposed for on-line surgical phase recognition from laparoscopic videos, modeling the inter-phase correlation implicitly by conditional probability distribution. To reduce inference bias and to enhance phase consistency, we further develop a consistency constraint inference strategy based on auto-regression. We conduct comprehensive validations on a well-known public dataset Cholec80. Experimental results show that our method outperforms the state-of-the-art methods both quantitatively and qualitatively, and achieves an inference rate of 66 frames per second (fps).