E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation

TL;DR

E2ENet is a novel 3D medical image segmentation model that combines dynamic sparse feature fusion and restricted depth-shift convolution to achieve high accuracy with significantly reduced computational resources.

Contribution

The paper introduces E2ENet, which employs a dynamic sparse feature fusion mechanism and a restricted depth-shift in 3D convolution to improve efficiency without sacrificing accuracy.

Findings

E2ENet outperforms prior methods in accuracy-efficiency trade-offs.

It reduces parameter count by over 68% and FLOPs by 29%.

Achieves comparable accuracy on large-scale challenges.

Abstract

Deep neural networks have evolved as the leading approach in 3D medical image segmentation due to their outstanding performance. However, the ever-increasing model size and computation cost of deep neural networks have become the primary barrier to deploying them on real-world resource-limited hardware. In pursuit of improving performance and efficiency, we propose a 3D medical image segmentation model, named Efficient to Efficient Network (E2ENet), incorporating two parametrically and computationally efficient designs. i. Dynamic sparse feature fusion (DSFF) mechanism: it adaptively learns to fuse informative multi-scale features while reducing redundancy. ii. Restricted depth-shift in 3D convolution: it leverages the 3D spatial information while keeping the model and computational complexity as 2D-based methods. We conduct extensive experiments on BTCV, AMOS-CT and Brain Tumor Segmentation Challenge, demonstrating that E2ENet consistently achieves a superior trade-off between accuracy and efficiency than prior arts across various resource constraints. E2ENet achieves comparable accuracy on the large-scale challenge AMOS-CT, while saving over 68\% parameter count and 29\% FLOPs in the inference phase, compared with the previous best-performing method. Our code has been made available at: https://github.com/boqian333/E2ENet-Medical.