Pursuing Open-Source Development of Predictive Algorithms: The Case of Criminal Sentencing Algorithms

TL;DR

Open-source development of criminal sentencing algorithms enhances transparency, accuracy, and cost-effectiveness, addressing biases and overfitting issues present in proprietary models, and should be adopted as the standard in high-stakes contexts.

Contribution

This paper advocates for open-source algorithms in criminal sentencing, demonstrating improved predictive accuracy and transparency over proprietary models through empirical replication and analysis.

Findings

Open-source algorithms show higher predictive power.

Replicated a major sentencing algorithm with real data.

Open-source models are more cost-effective and transparent.

Abstract

Currently, there is uncertainty surrounding the merits of open-source versus proprietary algorithm development. Though justification in favor of each exists, we argue that open-source algorithm development should be the standard in highly consequential contexts that affect people's lives for reasons of transparency and collaboration, which contribute to greater predictive accuracy and enjoy the additional advantage of cost-effectiveness. To make this case, we focus on criminal sentencing algorithms, as criminal sentencing is highly consequential, and impacts society and individual people. Further, the popularity of this topic has surged in the wake of recent studies uncovering racial bias in proprietary sentencing algorithms among other issues of over-fitting and model complexity. We suggest these issues are exacerbated by the proprietary and expensive nature of virtually all widely used criminal sentencing algorithms. Upon replicating a major algorithm using real criminal profiles, we fit three penalized regressions and demonstrate an increase in predictive power of these open-source and relatively computationally inexpensive options. The result is a data-driven suggestion that if judges who are making sentencing decisions want to craft appropriate sentences based on a high degree of accuracy and at low costs, then they should be pursuing open-source options.