Unequal Uncertainty: Rethinking Algorithmic Interventions for Mitigating Discrimination from AI

TL;DR

This paper critically examines how uncertainty-based algorithmic interventions in AI decision-making can unintentionally reinforce discrimination, and suggests that selective friction may promote fairness and accountability.

Contribution

It provides the first integrated socio-technical and legal analysis of uncertainty-based interventions, highlighting the potential of selective friction to mitigate discrimination in AI.

Findings

Selective abstention can worsen disparities for under-represented groups.

Both interventions may lead to unlawful discrimination under UK law.

Selective friction can enhance transparency and human judgment in AI decisions.

Abstract

Uncertainty in artificial intelligence (AI) predictions poses urgent legal and ethical challenges for AI-assisted decision-making. We examine two algorithmic interventions that act as guardrails for human-AI collaboration: selective abstention, which withholds high-uncertainty predictions from human decision-makers, and selective friction, which delivers those predictions together with salient warnings or disclosures that slow the decision process. Research has shown that selective abstention based on uncertainty can inadvertently exacerbate disparities and disadvantage under-represented groups that disproportionately receive uncertain predictions. In this paper, we provide the first integrated socio-technical and legal analysis of uncertainty-based algorithmic interventions. Through two case studies, AI-assisted consumer credit decisions and AI-assisted content moderation, we demonstrate how the seemingly neutral use of uncertainty thresholds can trigger discriminatory impacts. We argue that, although both interventions pose risks of unlawful discrimination under UK law, selective frictions offer a promising pathway toward fairer and more accountable AI-assisted decision-making by preserving transparency and encouraging more cautious human judgment.