CheXphotogenic: Generalization of Deep Learning Models for Chest X-ray Interpretation to Photos of Chest X-rays

TL;DR

This study evaluates how well deep learning models for chest X-ray interpretation perform on smartphone photos of X-rays, revealing performance drops but some models still match radiologist accuracy.

Contribution

It provides the first systematic assessment of deep learning model performance on photos of chest X-rays without additional tuning.

Findings

Several models' performance drops on photos of X-rays.

Some models still perform comparably to radiologists on photos.

Insights into how training procedures affect model generalization.

Abstract

The use of smartphones to take photographs of chest x-rays represents an appealing solution for scaled deployment of deep learning models for chest x-ray interpretation. However, the performance of chest x-ray algorithms on photos of chest x-rays has not been thoroughly investigated. In this study, we measured the diagnostic performance for 8 different chest x-ray models when applied to photos of chest x-rays. All models were developed by different groups and submitted to the CheXpert challenge, and re-applied to smartphone photos of x-rays in the CheXphoto dataset without further tuning. We found that several models had a drop in performance when applied to photos of chest x-rays, but even with this drop, some models still performed comparably to radiologists. Further investigation could be directed towards understanding how different model training procedures may affect model generalization to photos of chest x-rays.