TextRay: Mining Clinical Reports to Gain a Broad Understanding of Chest X-rays

TL;DR

TextRay leverages deep learning to analyze over two million chest X-ray reports, creating an ontology of common pathologies and outperforming radiologists in some diagnostic predictions, addressing the global shortage of radiologists.

Contribution

The paper introduces a large-scale ontology and a deep learning model trained on extensive CXR reports, improving automated interpretation and understanding of chest X-rays.

Findings

Deep learning model achieves high accuracy in predicting key findings.

Radiologists' agreement with the model exceeds inter-radiologist agreement for some diagnoses.

Constructed a comprehensive ontology of 40 prevalent CXR pathologies.

Abstract

The chest X-ray (CXR) is by far the most commonly performed radiological examination for screening and diagnosis of many cardiac and pulmonary diseases. There is an immense world-wide shortage of physicians capable of providing rapid and accurate interpretation of this study. A radiologist-driven analysis of over two million CXR reports generated an ontology including the 40 most prevalent pathologies on CXR. By manually tagging a relatively small set of sentences, we were able to construct a training set of 959k studies. A deep learning model was trained to predict the findings given the patient frontal and lateral scans. For 12 of the findings we compare the model performance against a team of radiologists and show that in most cases the radiologists agree on average more with the algorithm than with each other.