Large-scale Long-tailed Disease Diagnosis on Radiology Images

TL;DR

This paper presents RadDiag, a transformer-based model trained on a large publicly available radiology dataset, achieving high accuracy and versatility in diagnosing numerous diseases across multiple modalities, supporting zero-shot and fine-tuning applications.

Contribution

Introduces RadDiag, a novel foundation model for radiology diagnosis that leverages publicly available data and supports diverse modalities and diagnoses.

Findings

RadDiag achieves 95.14% AUC on internal evaluation.

RadDiag performs well in zero-shot and fine-tuning scenarios.

Public online medical data is valuable for building AI healthcare tools.

Abstract

Developing a generalist radiology diagnosis system can greatly enhance clinical diagnostics. In this paper, we introduce RadDiag, a foundational model supporting 2D and 3D inputs across various modalities and anatomies, using a transformer-based fusion module for comprehensive disease diagnosis. Due to patient privacy concerns and the lack of large-scale radiology diagnosis datasets, we utilize high-quality, clinician-reviewed radiological images available online with diagnosis labels. Our dataset, RP3D-DiagDS, contains 40,936 cases with 195,010 scans covering 5,568 disorders (930 unique ICD-10-CM codes). Experimentally, our RadDiag achieves 95.14% AUC on internal evaluation with the knowledge-enhancement strategy. Additionally, RadDiag can be zero-shot applied or fine-tuned to external diagnosis datasets sourced from various hospitals, demonstrating state-of-the-art results. In conclusion, we show that publicly shared medical data on the Internet is a tremendous and valuable resource that can potentially support building a generalist AI for healthcare.