Addressing Discretization-Induced Bias in Demographic Prediction

TL;DR

This paper investigates how discretization of continuous demographic predictions causes bias, particularly undercounting African-American voters, and proposes a joint optimization method to eliminate this bias with minimal accuracy loss.

Contribution

It introduces a novel joint optimization approach and a data-driven thresholding heuristic to address discretization bias in demographic prediction tasks.

Findings

Discretization bias can significantly undercount certain demographic groups.

The proposed method effectively reduces bias with negligible accuracy loss.

Calibrated continuous models alone are insufficient to eliminate discretization bias.

Abstract

Racial and other demographic imputation is necessary for many applications, especially in auditing disparities and outreach targeting in political campaigns. The canonical approach is to construct continuous predictions -- e.g., based on name and geography -- and then to $\textit{discretize}$ the predictions by selecting the most likely class (argmax). We study how this practice produces $\textit{discretization bias}$. In particular, we show that argmax labeling, as used by a prominent commercial voter file vendor to impute race/ethnicity, results in a substantial under-count of African-American voters, e.g., by 28.2% points in North Carolina. This bias can have substantial implications in downstream tasks that use such labels. We then introduce a $\textit{joint optimization}$ approach -- and a tractable $\textit{data-driven thresholding}$ heuristic -- that can eliminate this bias, with negligible individual-level accuracy loss. Finally, we theoretically analyze discretization bias, show that calibrated continuous models are insufficient to eliminate it, and that an approach such as ours is necessary. Broadly, we warn researchers and practitioners against discretizing continuous demographic predictions without considering downstream consequences.