Humanwashing -- It Should Leave You Feeling Dirty

TL;DR

The paper critically examines the metaphor of 'human in the loop' in AI, arguing it often obscures actual oversight processes and enables superficial 'humanwashing' rather than meaningful accountability.

Contribution

It challenges the effectiveness of the 'human in the loop' metaphor and highlights the need for clearer understanding of human oversight in AI systems.

Findings

The 'human in the loop' metaphor can obscure actual oversight processes.

Use of the metaphor enables 'humanwashing' to superficially improve AI system perception.

There is a need for more precise language and understanding of human oversight in AI.

Abstract

The phrase 'human in the loop' is increasingly used to imply a sense of safety in relation to AI decision systems. It shouldn't. There are contexts where it can be applied appropriately, but these are not in the deployed decision systems we see dominating today. Human oversight of AI decision processes is one of the most popular proposals for addressing concerns, especially about bias, discrimination, misinformation, manipulation, accountability, and transparency. But there is insufficient examination of what human oversight actually means. The question raised in this paper is whether using the metaphor of a loop does anything to assist understanding of what is required and what is achieved in a particular decision context. Indiscriminate use of the loop metaphor obscures both processes and outcomes. It enables 'humanwashing', an activity analogous to 'greenwashing', where writers and commentators use language primarily aimed at putting systems in the best possible light.