Testing for racial bias using inconsistent perceptions of race

TL;DR

This paper introduces a novel method to detect racial bias by examining how the same individual is treated differently based on perceived race, rather than direct comparisons between different individuals.

Contribution

The paper proposes a new bias detection test that uses perceptions of race to assess differential treatment, applicable to datasets with perceived identity data.

Findings

Drivers perceived as Hispanic are more likely to be searched or arrested than when perceived as white.

The method reveals racial bias in police traffic stops based on perceived race.

Applicable to datasets with perceived rather than self-reported race or gender.

Abstract

Tests for racial bias commonly assess whether two people of different races are treated differently. A fundamental challenge is that, because two people may differ in many ways, factors besides race might explain differences in treatment. Here, we propose a test for bias which circumvents the difficulty of comparing two people by instead assessing whether the $\textit{same person}$ is treated differently when their race is perceived differently. We apply our method to test for bias in police traffic stops, finding that the same driver is likelier to be searched or arrested by police when they are perceived as Hispanic than when they are perceived as white. Our test is broadly applicable to other datasets where race, gender, or other identity data are perceived rather than self-reported, and the same person is observed multiple times.