Contestable Black Boxes

TL;DR

This paper explores the importance of contestability in automated decision-making, emphasizing the need for specialized evaluation methods and proposing a socio-technical approach combining software engineering and rule-based techniques.

Contribution

It highlights the gap in AI research regarding contestability, proposing a new framework for evaluating and ensuring contestability in black-box algorithms.

Findings

Need for specialized methodologies to evaluate contestability.

Proposed socio-technical solution combining engineering and rule-based approaches.

Raises new questions about contestability versus explainability.

Abstract

The right to contest a decision with consequences on individuals or the society is a well-established democratic right. Despite this right also being explicitly included in GDPR in reference to automated decision-making, its study seems to have received much less attention in the AI literature compared, for example, to the right for explanation. This paper investigates the type of assurances that are needed in the contesting process when algorithmic black-boxes are involved, opening new questions about the interplay of contestability and explainability. We argue that specialised complementary methodologies to evaluate automated decision-making in the case of a particular decision being contested need to be developed. Further, we propose a combination of well-established software engineering and rule-based approaches as a possible socio-technical solution to the issue of contestability, one of the new democratic challenges posed by the automation of decision making.