Analyzing Fairness of Classification Machine Learning Model with Structured Dataset

TL;DR

This paper evaluates the fairness of classification machine learning models on structured datasets using three popular fairness libraries, comparing their effectiveness and providing practical guidance for reducing bias in real-world applications.

Contribution

It systematically compares three fairness libraries—Fairlearn, AIF360, and What If Tool—highlighting their strengths, limitations, and practical utility in mitigating bias in ML models.

Findings

Each library has unique strengths and limitations.

Fairness evaluation varies across different tools.

Practical guidance for integrating fairness tools into ML workflows.

Abstract

Machine learning (ML) algorithms have become integral to decision making in various domains, including healthcare, finance, education, and law enforcement. However, concerns about fairness and bias in these systems pose significant ethical and social challenges. This study investigates the fairness of ML models applied to structured datasets in classification tasks, highlighting the potential for biased predictions to perpetuate systemic inequalities. A publicly available dataset from Kaggle was selected for analysis, offering a realistic scenario for evaluating fairness in machine learning workflows. To assess and mitigate biases, three prominent fairness libraries; Fairlearn by Microsoft, AIF360 by IBM, and the What If Tool by Google were employed. These libraries provide robust frameworks for analyzing fairness, offering tools to evaluate metrics, visualize results, and implement bias mitigation strategies. The research aims to assess the extent of bias in the ML models, compare the effectiveness of these libraries, and derive actionable insights for practitioners. The findings reveal that each library has unique strengths and limitations in fairness evaluation and mitigation. By systematically comparing their capabilities, this study contributes to the growing field of ML fairness by providing practical guidance for integrating fairness tools into real world applications. These insights are intended to support the development of more equitable machine learning systems.