PSTNet: Enhanced Polyp Segmentation with Multi-scale Alignment and Frequency Domain Integration

TL;DR

PSTNet introduces a multi-scale, frequency-aware transformer-based network that significantly improves colorectal polyp segmentation accuracy by integrating frequency domain cues and addressing feature misalignment issues.

Contribution

The paper proposes PSTNet, a novel architecture that combines frequency domain information with multi-scale feature alignment for enhanced polyp segmentation.

Findings

Outperforms state-of-the-art methods on challenging datasets.

Demonstrates significant improvement in segmentation accuracy.

Effectively integrates frequency cues with semantic features.

Abstract

Accurate segmentation of colorectal polyps in colonoscopy images is crucial for effective diagnosis and management of colorectal cancer (CRC). However, current deep learning-based methods primarily rely on fusing RGB information across multiple scales, leading to limitations in accurately identifying polyps due to restricted RGB domain information and challenges in feature misalignment during multi-scale aggregation. To address these limitations, we propose the Polyp Segmentation Network with Shunted Transformer (PSTNet), a novel approach that integrates both RGB and frequency domain cues present in the images. PSTNet comprises three key modules: the Frequency Characterization Attention Module (FCAM) for extracting frequency cues and capturing polyp characteristics, the Feature Supplementary Alignment Module (FSAM) for aligning semantic information and reducing misalignment noise, and the Cross Perception localization Module (CPM) for synergizing frequency cues with high-level semantics to achieve efficient polyp segmentation. Extensive experiments on challenging datasets demonstrate PSTNet's significant improvement in polyp segmentation accuracy across various metrics, consistently outperforming state-of-the-art methods. The integration of frequency domain cues and the novel architectural design of PSTNet contribute to advancing computer-assisted polyp segmentation, facilitating more accurate diagnosis and management of CRC.