PHTrans: Parallelly Aggregating Global and Local Representations for Medical Image Segmentation

TL;DR

PHTrans introduces a parallel hybrid architecture combining CNNs and Transformers for medical image segmentation, effectively capturing global and local features to improve accuracy over existing methods.

Contribution

The paper proposes a novel parallel hybrid architecture, PHTrans, that simultaneously learns and aggregates global and local features in medical image segmentation.

Findings

Consistently outperforms state-of-the-art methods on multiple datasets.

Effective integration of global and local features improves segmentation accuracy.

Validated on Multi-Atlas Labeling and Cardiac Diagnosis datasets.

Abstract

The success of Transformer in computer vision has attracted increasing attention in the medical imaging community. Especially for medical image segmentation, many excellent hybrid architectures based on convolutional neural networks (CNNs) and Transformer have been presented and achieve impressive performance. However, most of these methods, which embed modular Transformer into CNNs, struggle to reach their full potential. In this paper, we propose a novel hybrid architecture for medical image segmentation called PHTrans, which parallelly hybridizes Transformer and CNN in main building blocks to produce hierarchical representations from global and local features and adaptively aggregate them, aiming to fully exploit their strengths to obtain better segmentation performance. Specifically, PHTrans follows the U-shaped encoder-decoder design and introduces the parallel hybird module in deep stages, where convolution blocks and the modified 3D Swin Transformer learn local features and global dependencies separately, then a sequence-to-volume operation unifies the dimensions of the outputs to achieve feature aggregation. Extensive experimental results on both Multi-Atlas Labeling Beyond the Cranial Vault and Automated Cardiac Diagnosis Challeng datasets corroborate its effectiveness, consistently outperforming state-of-the-art methods. The code is available at: https://github.com/lseventeen/PHTrans.