Uncovering the Source of Machine Bias

TL;DR

This paper develops a structural econometric model to identify and quantify gender biases in human evaluators' decisions on a micro-lending platform, showing biases favor female applicants and how removing them improves profits and reduces gender gaps.

Contribution

It introduces a novel econometric approach to uncover and simulate the effects of gender biases in lending decisions, and compares human biases with machine learning mitigation strategies.

Findings

Biases favor female applicants in lending decisions.

Removing biases increases company profits.

Machine learning can mitigate evaluator biases.

Abstract

We develop a structural econometric model to capture the decision dynamics of human evaluators on an online micro-lending platform, and estimate the model parameters using a real-world dataset. We find two types of biases in gender, preference-based bias and belief-based bias, are present in human evaluators' decisions. Both types of biases are in favor of female applicants. Through counterfactual simulations, we quantify the effect of gender bias on loan granting outcomes and the welfare of the company and the borrowers. Our results imply that both the existence of the preference-based bias and that of the belief-based bias reduce the company's profits. When the preference-based bias is removed, the company earns more profits. When the belief-based bias is removed, the company's profits also increase. Both increases result from raising the approval probability for borrowers, especially male borrowers, who eventually pay back loans. For borrowers, the elimination of either bias decreases the gender gap of the true positive rates in the credit risk evaluation. We also train machine learning algorithms on both the real-world data and the data from the counterfactual simulations. We compare the decisions made by those algorithms to see how evaluators' biases are inherited by the algorithms and reflected in machine-based decisions. We find that machine learning algorithms can mitigate both the preference-based bias and the belief-based bias.