Deep Doubly Supervised Transfer Network for Diagnosis of Breast Cancer with Imbalanced Ultrasound Imaging Modalities

TL;DR

This paper introduces a novel deep transfer learning network that effectively addresses modality imbalance in breast cancer ultrasound diagnosis by leveraging privileged information and MMD to improve knowledge transfer between ultrasound modalities.

Contribution

The proposed DDSTN uniquely combines LUPI and MMD in a deep transfer learning framework to better utilize label information and handle unpaired data in breast cancer ultrasound diagnosis.

Findings

DDSTN outperforms state-of-the-art algorithms on breast ultrasound datasets.

The method effectively utilizes privileged information to guide transfer learning.

Enhanced knowledge transfer improves diagnostic accuracy in imbalanced modality scenarios.

Abstract

Elastography ultrasound (EUS) provides additional bio-mechanical in-formation about lesion for B-mode ultrasound (BUS) in the diagnosis of breast cancers. However, joint utilization of both BUS and EUS is not popular due to the lack of EUS devices in rural hospitals, which arouses a novel modality im-balance problem in computer-aided diagnosis (CAD) for breast cancers. Current transfer learning (TL) pay little attention to this special issue of clinical modality imbalance, that is, the source domain (EUS modality) has fewer labeled samples than those in the target domain (BUS modality). Moreover, these TL methods cannot fully use the label information to explore the intrinsic relation between two modalities and then guide the promoted knowledge transfer. To this end, we propose a novel doubly supervised TL network (DDSTN) that integrates the Learning Using Privileged Information (LUPI) paradigm and the Maximum Mean Discrepancy (MMD) criterion into a unified deep TL framework. The proposed algorithm can not only make full use of the shared labels to effectively guide knowledge transfer by LUPI paradigm, but also perform additional super-vised transfer between unpaired data. We further introduce the MMD criterion to enhance the knowledge transfer. The experimental results on the breast ultra-sound dataset indicate that the proposed DDSTN outperforms all the compared state-of-the-art algorithms for the BUS-based CAD.