Multi-dimension Transformer with Attention-based Filtering for Medical Image Segmentation

TL;DR

This paper introduces MDT-AF, a novel multi-dimension transformer with attention-based filtering designed to improve medical image segmentation by enhancing feature representation and robustness against noise.

Contribution

It proposes a redesigned transformer architecture with attention-based filtering and multi-dimensional self-attention to better utilize spatial and channel information in medical images.

Findings

Achieves state-of-the-art performance on three benchmarks.

Effectively mitigates low signal-to-noise ratio issues.

Enhances feature representation through multi-dimensional attention.

Abstract

The accurate segmentation of medical images is crucial for diagnosing and treating diseases. Recent studies demonstrate that vision transformer-based methods have significantly improved performance in medical image segmentation, primarily due to their superior ability to establish global relationships among features and adaptability to various inputs. However, these methods struggle with the low signal-to-noise ratio inherent to medical images. Additionally, the effective utilization of channel and spatial information, which are essential for medical image segmentation, is limited by the representation capacity of self-attention. To address these challenges, we propose a multi-dimension transformer with attention-based filtering (MDT-AF), which redesigns the patch embedding and self-attention mechanism for medical image segmentation. MDT-AF incorporates an attention-based feature filtering mechanism into the patch embedding blocks and employs a coarse-to-fine process to mitigate the impact of low signal-to-noise ratio. To better capture complex structures in medical images, MDT-AF extends the self-attention mechanism to incorporate spatial and channel dimensions, enriching feature representation. Moreover, we introduce an interaction mechanism to improve the feature aggregation between spatial and channel dimensions. Experimental results on three public medical image segmentation benchmarks show that MDT-AF achieves state-of-the-art (SOTA) performance.