H-DenseFormer: An Efficient Hybrid Densely Connected Transformer for Multimodal Tumor Segmentation

TL;DR

H-DenseFormer is a hybrid neural network combining CNN and Transformer components, designed for efficient multimodal tumor segmentation with improved accuracy and reduced computational complexity, demonstrated on public datasets.

Contribution

The paper introduces a novel hybrid densely connected network that integrates a Transformer-based multi-path embedding and a lightweight Densely Connected Transformer block for multimodal tumor segmentation.

Findings

Outperforms state-of-the-art methods on HECKTOR21 and PI-CAI22 datasets.

Achieves higher segmentation accuracy with lower computational cost.

Demonstrates robustness to varying numbers of input modalities.

Abstract

Recently, deep learning methods have been widely used for tumor segmentation of multimodal medical images with promising results. However, most existing methods are limited by insufficient representational ability, specific modality number and high computational complexity. In this paper, we propose a hybrid densely connected network for tumor segmentation, named H-DenseFormer, which combines the representational power of the Convolutional Neural Network (CNN) and the Transformer structures. Specifically, H-DenseFormer integrates a Transformer-based Multi-path Parallel Embedding (MPE) module that can take an arbitrary number of modalities as input to extract the fusion features from different modalities. Then, the multimodal fusion features are delivered to different levels of the encoder to enhance multimodal learning representation. Besides, we design a lightweight Densely Connected Transformer (DCT) block to replace the standard Transformer block, thus significantly reducing computational complexity. We conduct extensive experiments on two public multimodal datasets, HECKTOR21 and PI-CAI22. The experimental results show that our proposed method outperforms the existing state-of-the-art methods while having lower computational complexity. The source code is available at https://github.com/shijun18/H-DenseFormer.