OSCARS: An Outlier-Sensitive Content-Based Radiography Retrieval System

TL;DR

OSCARS is a novel radiography retrieval system that enhances fine-grained, intra- and inter-class similarity detection by integrating outlier detection and a specialized quadruplet sampling strategy, improving retrieval relevance in noisy datasets.

Contribution

The paper introduces OSCARS, combining unsupervised outlier detection with a quadruplet sampling method and weighted metric learning for improved medical image retrieval.

Findings

Effective in distinguishing intra-class variations

Improves retrieval relevance on noisy datasets

Validated on public radiography datasets

Abstract

Improving the retrieval relevance on noisy datasets is an emerging need for the curation of a large-scale clean dataset in the medical domain. While existing methods can be applied for class-wise retrieval (aka. inter-class), they cannot distinguish the granularity of likeness within the same class (aka. intra-class). The problem is exacerbated on medical external datasets, where noisy samples of the same class are treated equally during training. Our goal is to identify both intra/inter-class similarities for fine-grained retrieval. To achieve this, we propose an Outlier-Sensitive Content-based rAdiologhy Retrieval System (OSCARS), consisting of two steps. First, we train an outlier detector on a clean internal dataset in an unsupervised manner. Then we use the trained detector to generate the anomaly scores on the external dataset, whose distribution will be used to bin intra-class variations. Second, we propose a quadruplet (a, p, nintra, ninter) sampling strategy, where intra-class negatives nintra are sampled from bins of the same class other than the bin anchor a belongs to, while niner are randomly sampled from inter-classes. We suggest a weighted metric learning objective to balance the intra and inter-class feature learning. We experimented on two representative public radiography datasets. Experiments show the effectiveness of our approach. The training and evaluation code can be found in https://github.com/XiaoyuanGuo/oscars.