AI, Opacity, and Personal Autonomy

TL;DR

This paper discusses how opaque AI decision algorithms in critical areas can undermine personal autonomy by preventing individuals from understanding and influencing decisions affecting their lives.

Contribution

It introduces a moral concern about opacity in AI algorithms, linking transparency to the preservation of personal autonomy in life-changing decisions.

Findings

Opaque algorithms can obstruct individuals from shaping their lives according to their goals.

The paper offers a causal account of transparency and opacity in AI decision-making.

It highlights new moral challenges and tools for ensuring transparency in AI systems.

Abstract

Advancements in machine learning have fuelled the popularity of using AI decision algorithms in procedures such as bail hearings (Feller et al. 2016), medical diagnoses (Rajkomar et al. 2018; Esteva et al. 2019) and recruitment (Heilweil 2019, Van Esch et al. 2019). Academic articles (Floridi et al. 2018), policy texts (HLEG 2019), and popularizing books (O'Neill 2016, Eubanks 2018) alike warn that such algorithms tend to be _opaque_: they do not provide explanations for their outcomes. Building on a causal account of transparency and opacity as well as recent work on the value of causal explanation (Lombrozo 2011, Hitchcock 2012), I formulate a moral concern for opaque algorithms that is yet to receive a systematic treatment in the literature: when such algorithms are used in life-changing decisions, they can obstruct us from effectively shaping our lives according to our goals and preferences, thus undermining our autonomy. I argue that this concern deserves closer attention as it furnishes the call for transparency in algorithmic decision-making with both new tools and new challenges.