Challenging the Machine: Contestability in Government AI Systems

TL;DR

This paper discusses the challenges and opportunities in making government AI decision systems contestable, emphasizing the need for actionable guidance to enable individuals to challenge automated decisions, based on a workshop held in early 2024.

Contribution

It provides a detailed analysis of the challenges in implementing contestability in government AI systems and offers recommendations from a multi-stakeholder workshop to develop practical guidance.

Findings

Identified key challenges in enabling contestability in government AI systems

Highlighted the importance of clear guidelines for challenge mechanisms

Proposed actionable steps for agencies to improve contestability

Abstract

In an October 2023 executive order (EO), President Biden issued a detailed but largely aspirational road map for the safe and responsible development and use of artificial intelligence (AI). The challenge for the January 24-25, 2024 workshop was to transform those aspirations regarding one specific but crucial issue -- the ability of individuals to challenge government decisions made about themselves -- into actionable guidance enabling agencies to develop, procure, and use genuinely contestable advanced automated decision-making systems. While the Administration has taken important steps since the October 2023 EO, the insights garnered from our workshop remain highly relevant, as the requirements for contestability of advanced decision-making systems are not yet fully defined or implemented. The workshop brought together technologists, members of government agencies and civil society organizations, litigators, and researchers in an intensive two-day meeting that examined the challenges that users, developers, and agencies faced in enabling contestability in light of advanced automated decision-making systems. To ensure a free and open flow of discussion, the meeting was held under a modified version of the Chatham House rule. Participants were free to use any information or details that they learned, but they may not attribute any remarks made at the meeting by the identity or the affiliation of the speaker. Thus, the workshop summary that follows anonymizes speakers and their affiliation. Where an identification of an agency, company, or organization is made, it is done from a public, identified resource and does not necessarily reflect statements made by participants at the workshop. This document is a report of that workshop, along with recommendations and explanatory material.