Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes

TL;DR

This paper introduces a novel, unobtrusive clinical decision support tool that seamlessly integrates into clinicians' routines, improving adoption and effectiveness by emphasizing subtlety and contextual fit.

Contribution

It presents a new design approach for DSTs inspired by 'Unremarkable Computing', emphasizing unobtrusiveness and contextual integration, with field evaluation results.

Findings

Clinicians are more receptive to unobtrusive DSTs.

Designing for 'unremarkableness' enhances adoption.

Lessons learned in prototyping critical AI systems.

Abstract

Clinical decision support tools (DST) promise improved healthcare outcomes by offering data-driven insights. While effective in lab settings, almost all DSTs have failed in practice. Empirical research diagnosed poor contextual fit as the cause. This paper describes the design and field evaluation of a radically new form of DST. It automatically generates slides for clinicians' decision meetings with subtly embedded machine prognostics. This design took inspiration from the notion of "Unremarkable Computing", that by augmenting the users' routines technology/AI can have significant importance for the users yet remain unobtrusive. Our field evaluation suggests clinicians are more likely to encounter and embrace such a DST. Drawing on their responses, we discuss the importance and intricacies of finding the right level of unremarkableness in DST design, and share lessons learned in prototyping critical AI systems as a situated experience.