Almost Politically Acceptable Criminal Justice Risk Assessment

TL;DR

This paper proposes a machine learning approach for criminal justice risk assessment that aims to produce politically acceptable, fairer results by training on White offenders and adjusting for racial bias claims, rather than seeking perfect accuracy and fairness.

Contribution

It introduces a novel method that adjusts traditional machine learning risk assessments to address political and social concerns about racial bias in criminal justice.

Findings

Risk assessments can be adjusted to be politically acceptable.

Training on White offenders and adjusting for other groups can mitigate bias claims.

The approach responds to stakeholder concerns without requiring perfect fairness.

Abstract

In criminal justice risk forecasting, one can prove that it is impossible to optimize accuracy and fairness at the same time. One can also prove that it is impossible optimize at once all of the usual group definitions of fairness. In the policy arena, one is left with tradeoffs about which many stakeholders will adamantly disagree. In this paper, we offer a different approach. We do not seek perfectly accurate and fair risk assessments. We seek politically acceptable risk assessments. We describe and apply to data on 300,000 offenders a machine learning approach that responds to many of the most visible charges of "racial bias." Regardless of whether such claims are true, we adjust our procedures to compensate. We begin by training the algorithm on White offenders only and computing risk with test data separately for White offenders and Black offenders. Thus, the fitted algorithm structure is exactly the same for both groups; the algorithm treats all offenders as if they are White. But because White and Black offenders can bring different predictors distributions to the white-trained algorithm, we provide additional adjustments if needed. Insofar are conventional machine learning procedures do not produce accuracy and fairness that some stakeholders require, it is possible to alter conventional practice to respond explicitly to many salient stakeholder claims even if they are unsupported by the facts. The results can be a politically acceptable risk assessment tools.