MNet: Rethinking 2D/3D Networks for Anisotropic Medical Image Segmentation

TL;DR

MNet introduces a novel mesh network that adaptively balances inter-slice and intra-slice features in 3D medical image segmentation, outperforming existing methods across multiple datasets.

Contribution

The paper proposes MNet, a new network that fuses multi-dimensional features within modules to better handle anisotropic medical images, improving segmentation accuracy.

Findings

Outperforms existing methods on four public datasets.

Balances sparse inter-slice and dense intra-slice information effectively.

Achieves higher segmentation accuracy and smoother organ contours.

Abstract

The nature of thick-slice scanning causes severe inter-slice discontinuities of 3D medical images, and the vanilla 2D/3D convolutional neural networks (CNNs) fail to represent sparse inter-slice information and dense intra-slice information in a balanced way, leading to severe underfitting to inter-slice features (for vanilla 2D CNNs) and overfitting to noise from long-range slices (for vanilla 3D CNNs). In this work, a novel mesh network (MNet) is proposed to balance the spatial representation inter axes via learning. 1) Our MNet latently fuses plenty of representation processes by embedding multi-dimensional convolutions deeply into basic modules, making the selections of representation processes flexible, thus balancing representation for sparse inter-slice information and dense intra-slice information adaptively. 2) Our MNet latently fuses multi-dimensional features inside each basic module, simultaneously taking the advantages of 2D (high segmentation accuracy of the easily recognized regions in 2D view) and 3D (high smoothness of 3D organ contour) representations, thus obtaining more accurate modeling for target regions. Comprehensive experiments are performed on four public datasets (CT\&MR), the results consistently demonstrate the proposed MNet outperforms the other methods. The code and datasets are available at: https://github.com/zfdong-code/MNet