Fairmetrics: An R package for group fairness evaluation

TL;DR

Fairmetrics is an R package that provides tools for evaluating various group fairness metrics in machine learning models, helping researchers identify and mitigate biases related to protected attributes.

Contribution

The paper introduces the fairmetrics R package, offering a comprehensive, user-friendly framework for assessing multiple group fairness criteria in ML models.

Findings

Supports evaluation of independence, separation, and sufficiency fairness metrics

Includes point and interval estimates for metrics

Demonstrates application with MIMIC-II dataset

Abstract

Fairness is a growing area of machine learning (ML) that focuses on ensuring models do not produce systematically biased outcomes for specific groups, particularly those defined by protected attributes such as race, gender, or age. Evaluating fairness is a critical aspect of ML model development, as biased models can perpetuate structural inequalities. The {fairmetrics} R package offers a user-friendly framework for rigorously evaluating numerous group-based fairness criteria, including metrics based on independence (e.g., statistical parity), separation (e.g., equalized odds), and sufficiency (e.g., predictive parity). Group-based fairness criteria assess whether a model is equally accurate or well-calibrated across a set of predefined groups so that appropriate bias mitigation strategies can be implemented. {fairmetrics} provides both point and interval estimates for multiple metrics through a convenient wrapper function and includes an example dataset derived from the Medical Information Mart for Intensive Care, version II (MIMIC-II) database (Goldberger et al., 2000; Raffa, 2016).