Deep Learning for Automated Classification of Tuberculosis-Related Chest X-Ray: Dataset Specificity Limits Diagnostic Performance Generalizability

TL;DR

This study demonstrates that deep learning models trained on tuberculosis chest X-ray datasets from one population often do not perform well on different populations, highlighting the challenge of generalizability in medical AI.

Contribution

The paper provides empirical evidence that dataset specificity limits the diagnostic performance of deep learning models for tuberculosis detection across different populations.

Findings

Model trained on one population's dataset underperforms on another population.

Dataset differences like image quality and disease severity affect model accuracy.

Generalizability issues must be addressed before clinical deployment.

Abstract

Machine learning has been an emerging tool for various aspects of infectious diseases including tuberculosis surveillance and detection. However, WHO provided no recommendations on using computer-aided tuberculosis detection software because of the small number of studies, methodological limitations, and limited generalizability of the findings. To quantify the generalizability of the machine-learning model, we developed a Deep Convolutional Neural Network (DCNN) model using a TB-specific CXR dataset of one population (National Library of Medicine Shenzhen No.3 Hospital) and tested it with non-TB-specific CXR dataset of another population (National Institute of Health Clinical Centers). The findings suggested that a supervised deep learning model developed by using the training dataset from one population may not have the same diagnostic performance in another population. Technical specification of CXR images, disease severity distribution, overfitting, and overdiagnosis should be examined before implementation in other settings.