Choosing an algorithmic fairness metric for an online marketplace: Detecting and quantifying algorithmic bias on LinkedIn

TL;DR

This paper proposes a fairness metric based on equal opportunity to detect and quantify algorithmic bias in online marketplaces, specifically applied to LinkedIn's recommendation algorithms, distinguishing algorithmic bias from human bias.

Contribution

It introduces a new fairness metric derived from economic discrimination literature and a framework to differentiate algorithmic bias from human bias in two-sided platforms.

Findings

The proposed metric effectively detects algorithmic bias related to gender.

Application to LinkedIn algorithms reveals measurable gender bias.

The framework helps isolate algorithmic bias from societal and human biases.

Abstract

In this paper, we derive an algorithmic fairness metric from the fairness notion of equal opportunity for equally qualified candidates for recommendation algorithms commonly used by two-sided marketplaces. We borrow from the economic literature on discrimination to arrive at a test for detecting bias that is solely attributable to the algorithm, as opposed to other sources such as societal inequality or human bias on the part of platform users. We use the proposed method to measure and quantify algorithmic bias with respect to gender of two algorithms used by LinkedIn, a popular online platform used by job seekers and employers. Moreover, we introduce a framework and the rationale for distinguishing algorithmic bias from human bias, both of which can potentially exist on a two-sided platform where algorithms make recommendations to human users. Finally, we discuss the shortcomings of a few other common algorithmic fairness metrics and why they do not capture the fairness notion of equal opportunity for equally qualified candidates.