Deep metric learning for multi-labelled radiographs

TL;DR

This paper introduces a deep metric learning approach for multi-labelled radiographs, enabling the measurement of radiological similarity based on shared abnormalities, validated on a large dataset with promising clustering and retrieval results.

Contribution

It proposes novel loss functions for multi-labelled and noisy radiograph data within a deep metric learning framework, and demonstrates effectiveness on a large-scale chest radiograph dataset.

Findings

The learned metric effectively clusters similar radiological abnormalities.

The approach distinguishes normal from abnormal radiographs.

Performance validated on over 745,000 radiographs.

Abstract

Many radiological studies can reveal the presence of several co-existing abnormalities, each one represented by a distinct visual pattern. In this article we address the problem of learning a distance metric for plain radiographs that captures a notion of "radiological similarity": two chest radiographs are considered to be similar if they share similar abnormalities. Deep convolutional neural networks (DCNs) are used to learn a low-dimensional embedding for the radiographs that is equipped with the desired metric. Two loss functions are proposed to deal with multi-labelled images and potentially noisy labels. We report on a large-scale study involving over 745,000 chest radiographs whose labels were automatically extracted from free-text radiological reports through a natural language processing system. Using 4,500 validated exams, we demonstrate that the methodology performs satisfactorily on clustering and image retrieval tasks. Remarkably, the learned metric separates normal exams from those having radiological abnormalities.