The Imbalanced User-AI Relationships as an Ethical Failure of Front-End Design in Healthcare AI

TL;DR

This paper highlights front-end ethical failures in healthcare AI, focusing on imbalanced user-AI relationships caused by design choices that limit user agency and understanding.

Contribution

It introduces the concept of asymmetric legibility and proposes design interventions to foster more participatory and balanced user-AI interactions in healthcare.

Findings

Design choices like default recommendations undermine user agency.

Suppressed uncertainty reduces transparency and trust.

Reciprocity as a design principle can improve user-AI relationships.

Abstract

Ethical discourse on AI in healthcare has focused predominantly on back-end concerns such as bias, fairness and explainability, while the front-end interface, where patients and clinicians actually encounter AI outputs, remains under explored. This paper identifies imbalanced user-AI relationships as a distinct class of front-end ethical failure: patients are rendered highly visible to AI systems through data inference, yet cannot understand, question or influence how they are represented. Through the concept of asymmetric legibility and a chat-based telemedicine case, we show how design choices e.g., default recommendations, restricted inputs and suppressed uncertainty, undermine agency, clinician judgment and human oversight even where systems are technically accurate. We propose reciprocity as a design orientation and offer interventions for more balanced, participatory user-AI relationships in healthcare.