A Semantically-Aware Relevance Measure for Content-Based Medical Image Retrieval Evaluation

TL;DR

This paper proposes a novel, knowledge graph-based relevance measure for evaluating content-based medical image retrieval, leveraging medical concepts from associated text to improve assessment accuracy without manual labels.

Contribution

It introduces a new relevance measure using knowledge graphs to evaluate medical image retrieval based on medical concepts extracted from text, addressing limitations of traditional metrics.

Findings

Effective relevance measure demonstrated on public dataset

Improves evaluation accuracy without manual labels

Utilizes medical concept relationships for better assessment

Abstract

Performance evaluation for Content-Based Image Retrieval (CBIR) remains a crucial but unsolved problem today especially in the medical domain. Various evaluation metrics have been discussed in the literature to solve this problem. Most of the existing metrics (e.g., precision, recall) are adapted from classification tasks which require manual labels as ground truth. However, such labels are often expensive and unavailable in specific thematic domains. Furthermore, medical images are usually associated with (radiological) case reports or annotated with descriptive captions in literature figures, such text contains information that can help to assess CBIR.Several researchers have argued that the medical concepts hidden in the text can serve as the basis for CBIR evaluation purpose. However, these works often consider these medical concepts as independent and isolated labels while in fact the subtle relationships between various concepts are neglected. In this work, we introduce the use of knowledge graphs to measure the distance between various medical concepts and propose a novel relevance measure for the evaluation of CBIR by defining an approximate matching-based relevance score between two sets of medical concepts which allows us to indirectly measure the similarity between medical images.We quantitatively demonstrate the effectiveness and feasibility of our relevance measure using a public dataset.