What is the Point of Fairness? Disability, AI and The Complexity of Justice

TL;DR

This paper critiques the reliance on fairness in AI for disability, highlighting how it can reinforce power imbalances and advocating for a justice-oriented approach informed by disability studies and critical data science.

Contribution

It presents two case studies in computer vision illustrating how fairness-focused AI can cause harm to disabled people and argues for a shift towards justice-based frameworks.

Findings

Fairness frameworks may reinforce existing power structures.

Current AI tools can harm marginalized disabled groups.

A justice-oriented approach can better address structural injustices.

Abstract

Work integrating conversations around AI and Disability is vital and valued, particularly when done through a lens of fairness. Yet at the same time, analyzing the ethical implications of AI for disabled people solely through the lens of a singular idea of "fairness" risks reinforcing existing power dynamics, either through reinforcing the position of existing medical gatekeepers, or promoting tools and techniques that benefit otherwise-privileged disabled people while harming those who are rendered outliers in multiple ways. In this paper we present two case studies from within computer vision - a subdiscipline of AI focused on training algorithms that can "see" - of technologies putatively intended to help disabled people but, through failures to consider structural injustices in their design, are likely to result in harms not addressed by a "fairness" framing of ethics. Drawing on disability studies and critical data science, we call on researchers into AI ethics and disability to move beyond simplistic notions of fairness, and towards notions of justice.