Evidence of What, for Whom? The Socially Contested Role of Algorithmic Bias in a Predictive Policing Tool

TL;DR

This study critically examines how different stakeholders interpret and utilize evidence of algorithmic bias in predictive policing, revealing diverse, often conflicting, social and political implications in Chicago's criminal justice context.

Contribution

It provides a qualitative analysis of the varied social roles and uses of algorithmic bias evidence in policing, emphasizing stakeholder perspectives and power dynamics.

Findings

Stakeholders use bias evidence to reform police policies.

Some reject algorithmic policing interventions.

Bias evidence is used to challenge or reinforce authority.

Abstract

This paper presents a critical, qualitative study of the social role of algorithmic bias in the context of the Chicago crime prediction algorithm, a predictive policing tool that forecasts when and where in the city crime is most likely to occur. Through interviews with 18 Chicago-area community organizations, academic researchers, and public sector actors, we show that stakeholders from different groups articulate diverse problem diagnoses of the tool's algorithmic bias, strategically using it as evidence to advance criminal justice interventions that align with stakeholders' positionality and political ends. Drawing inspiration from Catherine D'Ignazio's taxonomy of "refusing and using" data, we find that stakeholders use evidence of algorithmic bias to reform the policies around police patrol allocation; reject algorithm-based policing interventions; reframe crime as a structural rather than interpersonal problem; reveal data on authority figures in an effort to subvert their power; repair and heal families and communities; and, in the case of more powerful actors, to reaffirm their own authority or existing power structures. We identify the implicit assumptions and scope of these varied uses of algorithmic bias as evidence, showing that they require different (and sometimes conflicting) values about policing and AI. This divergence reflects long-standing tensions in the criminal justice reform landscape between the values of liberation and healing often centered by system-impacted communities and the values of surveillance and deterrence often instantiated in data-driven reform measures. We advocate for centering the interests and experiential knowledge of communities impacted by incarceration to ensure that evidence of algorithmic bias can serve as a device to challenge the status quo.