Surgical Scene Segmentation by Transformer With Asymmetric Feature Enhancement

TL;DR

This paper introduces a Transformer-based framework with asymmetric feature enhancement for surgical scene segmentation, effectively improving local information fusion and fine-grained structure recognition in complex surgical images.

Contribution

It proposes a novel Transformer framework with an Asymmetric Feature Enhancement module that addresses key challenges in surgical scene segmentation.

Findings

Outperforms state-of-the-art methods in multiple surgical segmentation tasks

Enhances local information and multi-scale feature fusion

Proves effectiveness in fine-grained structure recognition

Abstract

Surgical scene segmentation is a fundamental task for robotic-assisted laparoscopic surgery understanding. It often contains various anatomical structures and surgical instruments, where similar local textures and fine-grained structures make the segmentation a difficult task. Vision-specific transformer method is a promising way for surgical scene understanding. However, there are still two main challenges. Firstly, the absence of inner-patch information fusion leads to poor segmentation performance. Secondly, the specific characteristics of anatomy and instruments are not specifically modeled. To tackle the above challenges, we propose a novel Transformer-based framework with an Asymmetric Feature Enhancement module (TAFE), which enhances local information and then actively fuses the improved feature pyramid into the embeddings from transformer encoders by a multi-scale interaction attention strategy. The proposed method outperforms the SOTA methods in several different surgical segmentation tasks and additionally proves its ability of fine-grained structure recognition. Code is available at https://github.com/cyuan-sjtu/ViT-asym.