Hybrid(Transformer+CNN)-based Polyp Segmentation

TL;DR

This paper introduces a hybrid Transformer and CNN-based model for polyp segmentation in colonoscopy images, significantly improving accuracy and robustness against artifacts and boundary ambiguities compared to existing methods.

Contribution

A novel hybrid architecture combining Transformer and CNN components that enhances polyp segmentation accuracy and robustness in challenging endoscopic conditions.

Findings

Improved segmentation recall to 95.55%.

Achieved 0.07% higher accuracy than state-of-the-art.

Demonstrated robustness against common endoscopic artifacts.

Abstract

Colonoscopy is still the main method of detection and segmentation of colonic polyps, and recent advancements in deep learning networks such as U-Net, ResUNet, Swin-UNet, and PraNet have made outstanding performance in polyp segmentation. Yet, the problem is extremely challenging due to high variation in size, shape, endoscopy types, lighting, imaging protocols, and ill-defined boundaries (fluid, folds) of the polyps, rendering accurate segmentation a challenging and problematic task. To address these critical challenges in polyp segmentation, we introduce a hybrid (Transformer + CNN) model that is crafted to enhance robustness against evolving polyp characteristics. Our hybrid architecture demonstrates superior performance over existing solutions, particularly in addressing two critical challenges: (1) accurate segmentation of polyps with ill-defined margins through boundary-aware attention mechanisms, and (2) robust feature extraction in the presence of common endoscopic artifacts, including specular highlights, motion blur, and fluid occlusions. Quantitative evaluations reveal significant improvements in segmentation accuracy (Recall improved by 1.76%, i.e., 0.9555, accuracy improved by 0.07%, i.e., 0.9849) and artifact resilience compared to state-of-the-art polyp segmentation methods.