Case-based Similar Image Retrieval for Weakly Annotated Large Histopathological Images of Malignant Lymphoma Using Deep Metric Learning

TL;DR

This paper introduces a deep learning-based image retrieval method for histopathological lymphoma images, focusing on tumor regions and incorporating IHC staining patterns to improve similarity assessment.

Contribution

It presents a novel attention-based multiple instance learning approach combined with contrastive metric learning for more accurate case-based image retrieval in lymphoma histopathology.

Findings

Higher evaluation metrics than baseline methods

Effective focus on tumor-specific regions

Pathologists confirmed the relevance of IHC-based similarity

Abstract

In the present study, we propose a novel case-based similar image retrieval (SIR) method for hematoxylin and eosin (H&E)-stained histopathological images of malignant lymphoma. When a whole slide image (WSI) is used as an input query, it is desirable to be able to retrieve similar cases by focusing on image patches in pathologically important regions such as tumor cells. To address this problem, we employ attention-based multiple instance learning, which enables us to focus on tumor-specific regions when the similarity between cases is computed. Moreover, we employ contrastive distance metric learning to incorporate immunohistochemical (IHC) staining patterns as useful supervised information for defining appropriate similarity between heterogeneous malignant lymphoma cases. In the experiment with 249 malignant lymphoma patients, we confirmed that the proposed method exhibited higher evaluation measures than the baseline case-based SIR methods. Furthermore, the subjective evaluation by pathologists revealed that our similarity measure using IHC staining patterns is appropriate for representing the similarity of H&E-stained tissue images for malignant lymphoma.