Extracting and Learning Fine-Grained Labels from Chest Radiographs

TL;DR

This paper introduces a novel method for extracting and learning over 450 fine-grained labels from chest X-ray reports, enabling deep learning models to recognize detailed findings in radiographs.

Contribution

It presents a new approach combining vocabulary-driven extraction and dependency parsing to obtain detailed labels, and trains a deep model for fine-grained classification of chest X-ray images.

Findings

High accuracy in label extraction

Reliable learning of fine-grained labels

First model to recognize detailed findings with multiple modifiers

Abstract

Chest radiographs are the most common diagnostic exam in emergency rooms and intensive care units today. Recently, a number of researchers have begun working on large chest X-ray datasets to develop deep learning models for recognition of a handful of coarse finding classes such as opacities, masses and nodules. In this paper, we focus on extracting and learning fine-grained labels for chest X-ray images. Specifically we develop a new method of extracting fine-grained labels from radiology reports by combining vocabulary-driven concept extraction with phrasal grouping in dependency parse trees for association of modifiers with findings. A total of 457 fine-grained labels depicting the largest spectrum of findings to date were selected and sufficiently large datasets acquired to train a new deep learning model designed for fine-grained classification. We show results that indicate a highly accurate label extraction process and a reliable learning of fine-grained labels. The resulting network, to our knowledge, is the first to recognize fine-grained descriptions of findings in images covering over nine modifiers including laterality, location, severity, size and appearance.