Autonomy by Design: Preserving Human Autonomy in AI Decision-Support

TL;DR

This paper examines how AI decision-support systems impact domain-specific human autonomy, identifying risks like loss of competence and value shifts, and proposes design patterns to preserve autonomy in specialized fields.

Contribution

It introduces a framework and practical design patterns for creating AI support systems that maintain human domain-specific autonomy.

Findings

AI systems can erode domain-specific autonomy without reliable failure indicators.

Unconscious value shifts caused by AI can diminish authentic decision-making.

Design patterns like role specification and defeater mechanisms help preserve autonomy.

Abstract

AI systems increasingly support human decision-making across domains of professional, skill-based, and personal activity. While previous work has examined how AI might affect human autonomy globally, the effects of AI on domain-specific autonomy -- the capacity for self-governed action within defined realms of skill or expertise -- remain understudied. We analyze how AI decision-support systems affect two key components of domain-specific autonomy: skilled competence (the ability to make informed judgments within one's domain) and authentic value-formation (the capacity to form genuine domain-relevant values and preferences). By engaging with prior investigations and analyzing empirical cases across medical, financial, and educational domains, we demonstrate how the absence of reliable failure indicators and the potential for unconscious value shifts can erode domain-specific autonomy both immediately and over time. We then develop a constructive framework for autonomy-preserving AI support systems. We propose specific socio-technical design patterns -- including careful role specification, implementation of defeater mechanisms, and support for reflective practice -- that can help maintain domain-specific autonomy while leveraging AI capabilities. This framework provides concrete guidance for developing AI systems that enhance rather than diminish human agency within specialized domains of action.