Transformer-Enhanced Iterative Feedback Mechanism for Polyp Segmentation

TL;DR

This paper introduces FANetv2, an advanced encoder-decoder network with a feedback attention mechanism and text-guided features, achieving state-of-the-art accuracy in automated polyp segmentation from colonoscopy images.

Contribution

FANetv2 uniquely combines iterative feedback refinement and text-guided attribute integration for improved polyp segmentation and classification.

Findings

Achieves high dice similarity coefficients of 0.9186 and 0.9481 on two datasets.

Demonstrates low Hausdorff distances indicating precise segmentation.

Outperforms existing methods in polyp segmentation accuracy.

Abstract

Colorectal cancer (CRC) is the third most common cause of cancer diagnosed in the United States and the second leading cause of cancer-related death among both genders. Notably, CRC is the leading cause of cancer in younger men less than 50 years old. Colonoscopy is considered the gold standard for the early diagnosis of CRC. Skills vary significantly among endoscopists, and a high miss rate is reported. Automated polyp segmentation can reduce the missed rates, and timely treatment is possible in the early stage. To address this challenge, we introduce \textit{\textbf{\ac{FANetv2}}}, an advanced encoder-decoder network designed to accurately segment polyps from colonoscopy images. Leveraging an initial input mask generated by Otsu thresholding, FANetv2 iteratively refines its binary segmentation masks through a novel feedback attention mechanism informed by the mask predictions of previous epochs. Additionally, it employs a text-guided approach that integrates essential information about the number (one or many) and size (small, medium, large) of polyps to further enhance its feature representation capabilities. This dual-task approach facilitates accurate polyp segmentation and aids in the auxiliary classification of polyp attributes, significantly boosting the model's performance. Our comprehensive evaluations on the publicly available BKAI-IGH and CVC-ClinicDB datasets demonstrate the superior performance of FANetv2, evidenced by high dice similarity coefficients (DSC) of 0.9186 and 0.9481, along with low Hausdorff distances of 2.83 and 3.19, respectively. The source code for FANetv2 is available at https://github.com/xxxxx/FANetv2.