Average Profits of Prejudiced Algorithms

TL;DR

This paper analyzes how the choice of scoring algorithms affects a firm's success in hiring disadvantaged applicants, considering the influence of prejudiced training data and decision-maker bias.

Contribution

It provides theoretical guarantees on when a firm benefits more from one of two common scoring algorithms based on the level of prejudice in the training data.

Findings

Identifies conditions favoring the prejudiced or unbiased algorithm

Quantifies the impact of decision-maker prejudice on algorithm effectiveness

Offers sharp theoretical guarantees for algorithm selection

Abstract

We investigate the level of success a firm achieves depending on which of two common scoring algorithms is used to screen qualified applicants belonging to a disadvantaged group. Both algorithms are trained on data generated by a prejudiced decision-maker independently of the firm. One algorithm favors disadvantaged individuals, while the other algorithm exemplifies prejudice in the training data. We deliver sharp guarantees for when the firm finds more success with one algorithm over the other, depending on the prejudice level of the decision-maker.