Breast Lesion Diagnosis Using Static Images and Dynamic Video

TL;DR

This paper introduces a multi-modality deep learning model for breast tumor diagnosis that combines static images and dynamic videos, mimicking radiologists' diagnostic process and improving classification accuracy.

Contribution

It proposes a novel multi-modality approach that integrates static and dynamic ultrasound data guided by domain knowledge, enhancing lesion classification performance.

Findings

Achieved 90.0% AUC in benign/malignant classification.

Improved diagnostic accuracy over single-modality models.

Validated on a dataset of 897 ultrasound sets.

Abstract

Deep learning based Computer Aided Diagnosis (CAD) systems have been developed to treat breast ultrasound. Most of them focus on a single ultrasound imaging modality, either using representative static images or the dynamic video of a real-time scan. In fact, these two image modalities are complementary for lesion diagnosis. Dynamic videos provide detailed three-dimensional information about the lesion, while static images capture the typical sections of the lesion. In this work, we propose a multi-modality breast tumor diagnosis model to imitate the diagnosing process of radiologists, which learns the features of both static images and dynamic video and explores the potential relationship between the two modalities. Considering that static images are carefully selected by professional radiologists, we propose to aggregate dynamic video features under the guidance of domain knowledge from static images before fusing multi-modality features. Our work is validated on a breast ultrasound dataset composed of 897 sets of ultrasound images and videos. Experimental results show that our model boosts the performance of Benign/Malignant classification, achieving 90.0% in AUC and 81.7% in accuracy.