Joint Weakly and Semi-Supervised Deep Learning for Localization and Classification of Masses in Breast Ultrasound Images

TL;DR

This paper introduces a hybrid weakly and semi-supervised deep learning framework for breast ultrasound mass localization and classification, reducing annotation effort while maintaining high accuracy.

Contribution

It presents a systematic training scenario combining weakly and strongly annotated data, achieving comparable results with significantly less strongly annotated data.

Findings

Achieved similar localization accuracy with only 10 strongly annotated images compared to 800.

Adding weakly annotated images improved localization performance by 4.5%.

The method effectively reduces annotation effort while maintaining high performance.

Abstract

We propose a framework for localization and classification of masses in breast ultrasound (BUS) images. We have experimentally found that training convolutional neural network based mass detectors with large, weakly annotated datasets presents a non-trivial problem, while overfitting may occur with those trained with small, strongly annotated datasets. To overcome these problems, we use a weakly annotated dataset together with a smaller strongly annotated dataset in a hybrid manner. We propose a systematic weakly and semi-supervised training scenario with appropriate training loss selection. Experimental results show that the proposed method can successfully localize and classify masses with less annotation effort. The results trained with only 10 strongly annotated images along with weakly annotated images were comparable to results trained from 800 strongly annotated images, with the 95% confidence interval of difference -3.00%--5.00%, in terms of the correct localization (CorLoc) measure, which is the ratio of images with intersection over union with ground truth higher than 0.5. With the same number of strongly annotated images, additional weakly annotated images can be incorporated to give a 4.5% point increase in CorLoc, from 80.00% to 84.50% (with 95% confidence intervals 76.00%--83.75% and 81.00%--88.00%). The effects of different algorithmic details and varied amount of data are presented through ablative analysis.