Enhancing Radiological Diagnosis: A Collaborative Approach Integrating AI and Human Expertise for Visual Miss Correction

TL;DR

This paper introduces CoRaX, a collaborative AI system that integrates eye gaze data and radiology reports to improve chest radiograph diagnosis by identifying and correcting perceptual errors through human-AI collaboration.

Contribution

The study develops and evaluates CoRaX, a novel multimodal AI system that enhances diagnostic accuracy by pinpointing perceptual errors and facilitating collaboration with radiologists.

Findings

CoRaX corrected 21% of missed abnormalities in a simulated error dataset.

The system achieved a Referral-Usefulness score of 0.63, indicating accurate region predictions.

84% of CoRaX's interactions had a Total-Usefulness score above 0.40.

Abstract

Human-AI collaboration to identify and correct perceptual errors in chest radiographs has not been previously explored. This study aimed to develop a collaborative AI system, CoRaX, which integrates eye gaze data and radiology reports to enhance diagnostic accuracy in chest radiology by pinpointing perceptual errors and refining the decision-making process. Using public datasets REFLACX and EGD-CXR, the study retrospectively developed CoRaX, employing a large multimodal model to analyze image embeddings, eye gaze data, and radiology reports. The system's effectiveness was evaluated based on its referral-making process, the quality of referrals, and performance in collaborative diagnostic settings. CoRaX was tested on a simulated error dataset of 271 samples with 28% (93 of 332) missed abnormalities. The system corrected 21% (71 of 332) of these errors, leaving 7% (22 of 312) unresolved. The Referral-Usefulness score, indicating the accuracy of predicted regions for all true referrals, was 0.63 (95% CI 0.59, 0.68). The Total-Usefulness score, reflecting the diagnostic accuracy of CoRaX's interactions with radiologists, showed that 84% (237 of 280) of these interactions had a score above 0.40. In conclusion, CoRaX efficiently collaborates with radiologists to address perceptual errors across various abnormalities, with potential applications in the education and training of novice radiologists.