How May U.S. Courts Scrutinize Their Recidivism Risk Assessment Tools? Contextualizing AI Fairness Criteria on a Judicial Scrutiny-based Framework

TL;DR

This paper explores how U.S. courts evaluate AI-based recidivism risk tools by linking legal fairness principles with technical AI fairness criteria, proposing a new integrated framework for judicial scrutiny.

Contribution

It develops a novel framework combining legal scrutiny categories with technical fairness criteria, contextualized across multiple jurisdictions, to better understand judicial evaluation of AI tools.

Findings

Major technical fairness criteria relate to constitutional mandates.

Procedural fairness categories are relevant but incomplete for judicial scrutiny.

Legal policies often overlook individual fairness considerations.

Abstract

The AI/HCI and legal communities have developed largely independent conceptualizations of fairness. This conceptual difference hinders the potential incorporation of technical fairness criteria (e.g., procedural, group, and individual fairness) into sustainable policies and designs, particularly for high-stakes applications like recidivism risk assessment. To foster common ground, we conduct legal research to identify if and how technical AI conceptualizations of fairness surface in primary legal sources. We find that while major technical fairness criteria can be linked to constitutional mandates such as ``Due Process'' and ``Equal Protection'' thanks to judicial interpretation, several challenges arise when operationalizing them into concrete statutes/regulations. These policies often adopt procedural and group fairness but ignore the major technical criterion of individual fairness. Regarding procedural fairness, judicial ``scrutiny'' categories are relevant but may not fully capture how courts scrutinize the use of demographic features in potentially discriminatory government tools like RRA. Furthermore, some policies contradict each other on whether to apply procedural fairness to certain demographic features. Thus, we propose a new framework, integrating U.S. demographics-related legal scrutiny concepts and technical fairness criteria, and contextualize it in three other major AI-adopting jurisdictions (EU, China, and India).