Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directions

TL;DR

This paper surveys and experimentally evaluates various tricks used in deep learning-based medical image segmentation, providing a comprehensive guide and open-source toolkit to improve future research and address key challenges.

Contribution

It systematically collects and tests MedISeg tricks across implementation phases, clarifies their effects, and offers an open-source plug-and-play repository for the community.

Findings

Certain tricks significantly improve segmentation accuracy.

Open-source toolkit facilitates easy integration of tricks.

Guides for addressing dataset size, class imbalance, and domain shifts.

Abstract

Over the past few years, the rapid development of deep learning technologies for computer vision has significantly improved the performance of medical image segmentation (MedISeg). However, the diverse implementation strategies of various models have led to an extremely complex MedISeg system, resulting in a potential problem of unfair result comparisons. In this paper, we collect a series of MedISeg tricks for different model implementation phases (i.e., pre-training model, data pre-processing, data augmentation, model implementation, model inference, and result post-processing), and experimentally explore the effectiveness of these tricks on consistent baselines. With the extensive experimental results on both the representative 2D and 3D medical image datasets, we explicitly clarify the effect of these tricks. Moreover, based on the surveyed tricks, we also open-sourced a strong MedISeg repository, where each component has the advantage of plug-and-play. We believe that this milestone work not only completes a comprehensive and complementary survey of the state-of-the-art MedISeg approaches, but also offers a practical guide for addressing the future medical image processing challenges including but not limited to small dataset, class imbalance learning, multi-modality learning, and domain adaptation. The code and training weights have been released at: https://github.com/hust-linyi/seg_trick.