Towards the Right Kind of Fairness in AI

TL;DR

This paper reviews various fairness metrics in AI, introduces the 'Fairness Compass' tool to help select appropriate fairness definitions based on context, and aims to improve transparency and trust in AI systems.

Contribution

It structures existing fairness metrics, explains their options through examples, and proposes the 'Fairness Compass' as a practical tool for selecting suitable fairness measures.

Findings

The 'Fairness Compass' simplifies fairness metric selection.

Provides a structured overview of fairness definitions.

Enhances transparency in fairness decision-making.

Abstract

Fairness is a concept of justice. Various definitions exist, some of them conflicting with each other. In the absence of an uniformly accepted notion of fairness, choosing the right kind for a specific situation has always been a central issue in human history. When it comes to implementing sustainable fairness in artificial intelligence systems, this old question plays a key role once again: How to identify the most appropriate fairness metric for a particular application? The answer is often a matter of context, and the best choice depends on ethical standards and legal requirements. Since ethics guidelines on this topic are kept rather general for now, we aim to provide more hands-on guidance with this document. Therefore, we first structure the complex landscape of existing fairness metrics and explain the different options by example. Furthermore, we propose the "Fairness Compass", a tool which formalises the selection process and makes identifying the most appropriate fairness definition for a given system a simple, straightforward procedure. Because this process also allows to document the reasoning behind the respective decisions, we argue that this approach can help to build trust from the user through explaining and justifying the implemented fairness.