Can LLMs Synthesize Court-Ready Statistical Evidence? Evaluating AI-Assisted Sentencing Bias Analysis for California Racial Justice Act Claims

TL;DR

This paper introduces Redo.io, an open-source platform that uses large language models to synthesize statistical evidence of racial bias in California sentencing, aiding legal challenges under the Racial Justice Act.

Contribution

It develops an LLM-powered interpretive layer that translates statistical test results into court-ready narratives, enhancing evidence presentation for racial bias claims.

Findings

LLM performance comparable to statisticians in evidence synthesis

AI-assisted analysis improves real-time evidence generation

Reduces legal backlog of resentencing opportunities

Abstract

Resentencing in California remains a complex legal challenge despite legislative reforms like the Racial Justice Act (2020), which allows defendants to challenge convictions based on statistical evidence of racial disparities in sentencing and charging. Policy implementation lags behind legislative intent, creating a 'second-chance gap' where hundreds of resentencing opportunities remain unidentified. We present Redo.io, an open-source platform that processes 95,000 prison records acquired under the California Public Records Act (CPRA) and generates court-ready statistical evidence of racial bias in sentencing for prima facie and discovery motions. We explore the design of an LLM-powered interpretive layer that synthesizes results from statistical methods like Odds Ratio, Relative Risk, and Chi-Square Tests into cohesive narratives contextualized with confidence intervals, sample sizes, and data limitations. Our evaluations comparing LLM performance to statisticians using the LLM-as-a-Judge framework suggest that AI can serve as a powerful descriptive assistant for real-time evidence generation when ethically incorporated in the analysis pipeline.