A Human-in-the-loop Framework to Construct Context-aware Mathematical Notions of Outcome Fairness

TL;DR

This paper introduces a human-in-the-loop framework that learns context-aware fairness notions by eliciting human judgments, aiming to incorporate moral and contextual considerations into mathematical fairness models for decision-making.

Contribution

It presents a novel framework that learns context-sensitive fairness formulations through human responses, extending economic models of Equality of Opportunity and encompassing existing fairness notions.

Findings

Framework successfully captures human fairness assessments.

Experiments demonstrate the approach's ability to adapt fairness notions.

Initial step towards stakeholder-driven fairness in machine learning.

Abstract

Existing mathematical notions of fairness fail to account for the context of decision-making. We argue that moral consideration of contextual factors is an inherently human task. So we present a framework to learn context-aware mathematical formulations of fairness by eliciting people's situated fairness assessments. Our family of fairness notions corresponds to a new interpretation of economic models of Equality of Opportunity (EOP), and it includes most existing notions of fairness as special cases. Our human-in-the-loop approach is designed to learn the appropriate parameters of the EOP family by utilizing human responses to pair-wise questions about decision subjects' circumstance and deservingness, and the harm/benefit imposed on them. We illustrate our framework in a hypothetical criminal risk assessment scenario by conducting a series of human-subject experiments on Amazon Mechanical Turk. Our work takes an important initial step toward empowering stakeholders to have a voice in the formulation of fairness for Machine Learning.