Misfitting With AI: How Blind People Verify and Contest AI Errors

TL;DR

This study explores how blind users verify and contest errors in AI visual assistance tools, revealing unique challenges and strategies that inform accessible explainable AI design.

Contribution

It provides an in-depth qualitative analysis of blind people's verification experiences with AI VAT, highlighting their strategies and needs for accessible explainability features.

Findings

Blind users struggle with complex layouts, languages, and cultural artifacts in AI VAT.

Users employ non-visual skills and cross-reference devices to verify AI output.

Design opportunities include affordances for contestation and supporting diverse verification strategies.

Abstract

Blind people use artificial intelligence-enabled visual assistance technologies (AI VAT) to gain visual access in their everyday lives, but these technologies are embedded with errors that may be difficult to verify non-visually. Previous studies have primarily explored sighted users' understanding of AI output and created vision-dependent explainable AI (XAI) features. We extend this body of literature by conducting an in-depth qualitative study with 26 blind people to understand their verification experiences and preferences. We begin by describing errors blind people encounter, highlighting how AI VAT fails to support complex document layouts, diverse languages, and cultural artifacts. We then illuminate how blind people make sense of AI through experimenting with AI VAT, employing non-visual skills, strategically including sighted people, and cross-referencing with other devices. Participants provided detailed opportunities for designing accessible XAI, such as affordances to support contestation. Informed by disability studies framework of misfitting and fitting, we unpacked harmful assumptions with AI VAT, underscoring the importance of celebrating disabled ways of knowing. Lastly, we offer practical takeaways for Responsible AI practice to push the field of accessible XAI forward.