De-Biasing Models of Biased Decisions: A Comparison of Methods Using Mortgage Application Data

TL;DR

This study demonstrates how machine learning models can inherit and replicate bias in mortgage decision data and compares several de-biasing methods, including a novel approach, to mitigate this bias.

Contribution

It introduces a new de-biasing method that takes the most favorable prediction over protected variables and compares it with existing techniques using mortgage data.

Findings

De-biasing can partially recover original decisions.

Bias via proxies affects de-biasing effectiveness.

The novel method shows promise in reducing bias.

Abstract

Prediction models can improve efficiency by automating decisions such as the approval of loan applications. However, they may inherit bias against protected groups from the data they are trained on. This paper adds counterfactual (simulated) ethnic bias to real data on mortgage application decisions, and shows that this bias is replicated by a machine learning model (XGBoost) even when ethnicity is not used as a predictive variable. Next, several other de-biasing methods are compared: averaging over prohibited variables, taking the most favorable prediction over prohibited variables (a novel method), and jointly minimizing errors as well as the association between predictions and prohibited variables. De-biasing can recover some of the original decisions, but the results are sensitive to whether the bias is effected through a proxy.