AI Failure Loops in Devalued Work: The Confluence of Overconfidence in AI and Underconfidence in Worker Expertise

TL;DR

This paper explores how systemic devaluation of worker expertise in feminized labor leads to AI failure loops, where overconfidence in AI and underconfidence in workers' skills perpetuate ineffective AI deployment and governance.

Contribution

It introduces the concept of AI Failure Loops, highlighting the impact of societal devaluation of certain labor sectors on AI design and evaluation.

Findings

AI deployments often ignore worker expertise in feminized labor.

Devaluation of skills leads to AI that fails to add value and diminishes worker visibility.

Systemic biases contribute to ongoing AI deployment failures in devalued occupations.

Abstract

A growing body of literature has focused on understanding and addressing workplace AI design failures. However, past work has largely overlooked the role of the devaluation of worker expertise in shaping the dynamics of AI development and deployment. In this paper, we examine the case of feminized labor: a class of devalued occupations historically misnomered as ``women's work,'' such as social work, K-12 teaching, and home healthcare. Drawing on literature on AI deployments in feminized labor contexts, we conceptualize AI Failure Loops: a set of interwoven, socio-technical failure modes that help explain how the systemic devaluation of workers' expertise negatively impacts, and is impacted by, AI design, evaluation, and governance practices. These failures demonstrate how misjudgments on the automatability of workers' skills can lead to AI deployments that fail to bring value to workers and, instead, further diminish the visibility of workers' expertise. We discuss research and design implications for workplace AI, especially for devalued occupations.