Pyramid Medical Transformer for Medical Image Segmentation
Zhuangzhuang Zhang, Weixiong Zhang

TL;DR
This paper introduces Pyramid Medical Transformer (PMTrans), a novel multi-scale attention model that combines CNN features and adaptive partitioning to improve medical image segmentation efficiency and accuracy.
Contribution
The paper proposes a pyramidal architecture with multi-resolution attention and adaptive partitioning, addressing computational costs and relation modeling limitations of existing transformers in medical imaging.
Findings
PMTrans outperforms recent CNN and transformer models on multiple datasets.
The multi-scale approach effectively captures long-range dependencies.
Adaptive partitioning improves relation modeling and computational efficiency.
Abstract
Deep neural networks have been a prevailing technique in the field of medical image processing. However, the most popular convolutional neural networks (CNNs) based methods for medical image segmentation are imperfect because they model long-range dependencies by stacking layers or enlarging filters. Transformers and the self-attention mechanism are recently proposed to effectively learn long-range dependencies by modeling all pairs of word-to-word attention regardless of their positions. The idea has also been extended to the computer vision field by creating and treating image patches as embeddings. Considering the computation complexity for whole image self-attention, current transformer-based models settle for a rigid partitioning scheme that potentially loses informative relations. Besides, current medical transformers model global context on full resolution images, leading to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Brain Tumor Detection and Classification · Medical Image Segmentation Techniques
MethodsMulti-Head Attention · Linear Layer · Absolute Position Encodings · Position-Wise Feed-Forward Layer · Residual Connection · Layer Normalization · Adam · Softmax · Label Smoothing · Byte Pair Encoding
