"It depends": Configuring AI to Improve Clinical Usefulness Across Contexts

TL;DR

This paper investigates how to configure AI systems in radiology to enhance their clinical usefulness across diverse real-world contexts by identifying key dependencies and technical dimensions.

Contribution

It introduces a framework of four technical dimensions and provides design recommendations for adapting AI to various clinical settings.

Findings

Identified 10 sociotechnical dependencies crucial for AI design in radiology.

Conceptualized four technical dimensions: AI functionality, focus, decision threshold, and explainability.

Provided four design recommendations for context-specific AI configuration.

Abstract

Artificial Intelligence (AI) repeatedly match or outperform radiologists in lab experiments. However, real-world implementations of radiological AI-based systems are found to provide little to no clinical value. This paper explores how to design AI for clinical usefulness in different contexts. We conducted 19 design sessions and design interventions with 13 radiologists from 7 clinical sites in Denmark and Kenya, based on three iterations of a functional AI-based prototype. Ten sociotechnical dependencies were identified as crucial for the design of AI in radiology. We conceptualised four technical dimensions that must be configured to the intended clinical context of use: AI functionality, AI medical focus, AI decision threshold, and AI Explainability. We present four design recommendations on how to address dependencies pertaining to the medical knowledge, clinic type, user expertise level, patient context, and user situation that condition the configuration of these technical dimensions.