The Limits of Computation in Solving Equity Trade-Offs in Machine Learning and Justice System Risk Assessment

TL;DR

This paper examines the computational challenges and limitations of using machine learning for equitable risk assessment in the justice system, highlighting inherent trade-offs and the need for value-driven solutions.

Contribution

It analyzes the computational limits of achieving racial equity in justice-related machine learning models and emphasizes the importance of values beyond computation.

Findings

Models show different score distributions across racial groups.

Computational methods cannot fully resolve trade-offs in equity.

Values are necessary to address unresolved issues.

Abstract

This paper explores how different ideas of racial equity in machine learning, in justice settings in particular, can present trade-offs that are difficult to solve computationally. Machine learning is often used in justice settings to create risk assessments, which are used to determine interventions, resources, and punitive actions. Overall aspects and performance of these machine learning-based tools, such as distributions of scores, outcome rates by levels, and the frequency of false positives and true positives, can be problematic when examined by racial group. Models that produce different distributions of scores or produce a different relationship between level and outcome are problematic when those scores and levels are directly linked to the restriction of individual liberty and to the broader context of racial inequity. While computation can help highlight these aspects, data and computation are unlikely to solve them. This paper explores where values and mission might have to fill the spaces computation leaves.