2-Factor Retrieval for Improved Human-AI Decision Making in Radiology

TL;DR

This paper introduces '2-factor retrieval', a novel explainable AI technique for radiology that improves clinician accuracy by combining interface design with data retrieval, especially aiding less confident radiologists.

Contribution

The study proposes a new explainable AI method called 2-factor retrieval that enhances decision-making in radiology by allowing clinicians to verify AI suggestions through similar images.

Findings

2FR increases clinician accuracy in chest X-ray diagnosis.

Radiologists with low confidence benefit most from 2FR.

Compared to saliency and Shapley values, 2FR offers better verification.

Abstract

Human-machine teaming in medical AI requires us to understand to what degree a trained clinician should weigh AI predictions. While previous work has shown the potential of AI assistance at improving clinical predictions, existing clinical decision support systems either provide no explainability of their predictions or use techniques like saliency and Shapley values, which do not allow for physician-based verification. To address this gap, this study compares previously used explainable AI techniques with a newly proposed technique termed '2-factor retrieval (2FR)', which is a combination of interface design and search retrieval that returns similarly labeled data without processing this data. This results in a 2-factor security blanket where: (a) correct images need to be retrieved by the AI; and (b) humans should associate the retrieved images with the current pathology under test. We find that when tested on chest X-ray diagnoses, 2FR leads to increases in clinician accuracy, with particular improvements when clinicians are radiologists and have low confidence in their decision. Our results highlight the importance of understanding how different modes of human-AI decision making may impact clinician accuracy in clinical decision support systems.