BreastSegNet: Multi-label Segmentation of Breast MRI

TL;DR

This paper introduces BreastSegNet, a comprehensive multi-label segmentation method for breast MRI that covers nine anatomical structures, supported by a large annotated dataset and benchmarking of nine models, with nnU-Net ResEncM achieving the best results.

Contribution

The study presents a new multi-label segmentation algorithm for breast MRI, a large annotated dataset, and a benchmarking of nine models to advance quantitative breast imaging analysis.

Findings

nnU-Net ResEncM achieves highest Dice score of 0.694 overall.

Model performs best on heart, liver, muscle, FGT, and bone.

Dice scores exceed 0.73 for several key structures.

Abstract

Breast MRI provides high-resolution imaging critical for breast cancer screening and preoperative staging. However, existing segmentation methods for breast MRI remain limited in scope, often focusing on only a few anatomical structures, such as fibroglandular tissue or tumors, and do not cover the full range of tissues seen in scans. This narrows their utility for quantitative analysis. In this study, we present BreastSegNet, a multi-label segmentation algorithm for breast MRI that covers nine anatomical labels: fibroglandular tissue (FGT), vessel, muscle, bone, lesion, lymph node, heart, liver, and implant. We manually annotated a large set of 1123 MRI slices capturing these structures with detailed review and correction from an expert radiologist. Additionally, we benchmark nine segmentation models, including U-Net, SwinUNet, UNet++, SAM, MedSAM, and nnU-Net with multiple ResNet-based encoders. Among them, nnU-Net ResEncM achieves the highest average Dice scores of 0.694 across all labels. It performs especially well on heart, liver, muscle, FGT, and bone, with Dice scores exceeding 0.73, and approaching 0.90 for heart and liver. All model code and weights are publicly available, and we plan to release the data at a later date.