The Likelihood Ratio Wall: Structural Limits on Accurate Risk Assessment for Rare Violence

TL;DR

This paper establishes fundamental statistical limits on the accuracy of risk assessment tools for rare violent re-offense prediction, highlighting inherent challenges and implications for fairness and policy.

Contribution

It introduces the Likelihood Ratio Wall and Surveillance Ceiling, theoretical bounds demonstrating the limitations of current risk assessment methods for rare violent outcomes.

Findings

Achieving high PPV for rare violence is statistically impossible with current tools.

Post-hoc recalibration cannot overcome the fundamental discrimination limits.

Over-policing inflates risk factors, reducing maximum achievable precision for certain groups.

Abstract

Pretrial risk assessment tools are used on over one million U.S. defendants each year, yet their use for predicting rare violent re-offense faces a basic statistical barrier. We derive a universal precision bound -- the Likelihood Ratio Wall -- showing that when violent re-arrest rates are low (2-5%), achieving even a 50% hit rate among people labeled "high risk" (positive predictive value, or PPV) would require tools far more discriminative than current instruments appear to be. For rare outcomes, a tool can have respectable-looking performance metrics and still be wrong most of the time it flags someone as "high risk for violence." We show that post-hoc score recalibration cannot solve this problem because it does not improve the tool's underlying ability to separate true positives from false positives. We further prove a Surveillance Ceiling: when over-policing inflates recorded "risk factors" among those who would not re-offend, the maximum achievable precision is structurally lower for over-policed groups, even at equal offense rates. We translate these results into the Number Needed to Detain (how many people must be detained to prevent one violent offense), and propose that risk reports should communicate this uncertainty explicitly. Our findings suggest that for rare violent outcomes, debates about fairness metrics alone are incomplete: under current data regimes, the available features may not support high-confidence individualized detention decisions.