Uncertainty in Criminal Justice Algorithms: simulation studies of the Pennsylvania Additive Classification Tool

TL;DR

This paper investigates the Pennsylvania Additive Classification Tool (PACT), analyzing its fairness, feature importance, and outcome variability through simulations to understand the propagation of uncertainty in carceral algorithms.

Contribution

It introduces simulation-based methods to study the uncertainty and fairness of carceral algorithms like PACT, extending beyond traditional outcome analysis.

Findings

PACT's fairness varies across sex, age, and race.

Outcome variability can propagate uncertainty in repeated algorithm use.

Simulation reveals potential biases and instability in custody level assignments.

Abstract

Much attention has been paid to algorithms related to sentencing, the setting of bail, parole decisions and recidivism while less attention has been paid to carceral algorithms, those algorithms used to determine an incarcerated individual's lived experience. In this paper we study one such algorithm, the Pennsylvania Additive Classification Tool (PACT) that assigns custody levels to incarcerated individuals. We analyze the PACT in ways that criminal justice algorithms are often analyzed: namely, we train an accurate machine learning model for the PACT; we study its fairness across sex, age and race; and we determine which features are most important. In addition to these conventional computations, we propose and carry out some new ways to study such algorithms. Instead of focusing on the outcomes themselves, we propose shifting our attention to the variability in the outcomes, especially because many carceral algorithms are used repeatedly and there can be a propagation of uncertainty. By carrying out several simulations of assigning custody levels, we shine light on problematic aspects of tools like the PACT.