Developer Insights into Designing AI-Based Computer Perception Tools
Maya Guhan (1), Meghan E. Hurley (1), Eric A. Storch (2), John Herrington (3), Casey Zampella (3), Julia Parish-Morris (3), Gabriel L\'azaro-Mu\~noz (4), and Kristin Kostick-Quenet (1) ((1) Center for Ethics, Health Policy, Baylor College of Medicine, Houston, TX, USA

TL;DR
This study explores how developers of AI-based computer perception tools balance clinical utility, user trust, and ethical considerations, emphasizing design priorities like explainability, workflow integration, and stakeholder customization.
Contribution
It provides insights from interviews with developers, highlighting key design priorities and ethical considerations for integrating AI tools into clinical practice.
Findings
Developers prioritize explainability and context-awareness.
Alignment with clinical workflows is essential.
Ethical stewardship influences design choices.
Abstract
Artificial intelligence (AI)-based computer perception (CP) technologies use mobile sensors to collect behavioral and physiological data for clinical decision-making. These tools can reshape how clinical knowledge is generated and interpreted. However, effective integration of these tools into clinical workflows depends on how developers balance clinical utility with user acceptability and trustworthiness. Our study presents findings from 20 in-depth interviews with developers of AI-based CP tools. Interviews were transcribed and inductive, thematic analysis was performed to identify 4 key design priorities: 1) to account for context and ensure explainability for both patients and clinicians; 2) align tools with existing clinical workflows; 3) appropriately customize to relevant stakeholders for usability and acceptability; and 4) push the boundaries of innovation while aligning with…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsArtificial Intelligence in Healthcare and Education · Explainable Artificial Intelligence (XAI) · Electronic Health Records Systems
