Applications and Challenges of Fairness APIs in Machine Learning Software

TL;DR

This paper provides a qualitative analysis of how open-source fairness APIs are used in real-world machine learning applications, highlighting challenges faced by developers and the need for better education and resources.

Contribution

It offers insights into the usage scenarios, challenges, and developer needs related to fairness APIs in machine learning, based on an analysis of 204 GitHub repositories.

Findings

APIs are used mainly for learning and solving real-world problems

Developers face troubleshooting issues and lack expertise in bias mitigation

There are 17 distinct use-cases identified for fairness APIs

Abstract

Machine Learning software systems are frequently used in our day-to-day lives. Some of these systems are used in various sensitive environments to make life-changing decisions. Therefore, it is crucial to ensure that these AI/ML systems do not make any discriminatory decisions for any specific groups or populations. In that vein, different bias detection and mitigation open-source software libraries (aka API libraries) are being developed and used. In this paper, we conduct a qualitative study to understand in what scenarios these open-source fairness APIs are used in the wild, how they are used, and what challenges the developers of these APIs face while developing and adopting these libraries. We have analyzed 204 GitHub repositories (from a list of 1885 candidate repositories) which used 13 APIs that are developed to address bias in ML software. We found that these APIs are used for two primary purposes (i.e., learning and solving real-world problems), targeting 17 unique use-cases. Our study suggests that developers are not well-versed in bias detection and mitigation; they face lots of troubleshooting issues, and frequently ask for opinions and resources. Our findings can be instrumental for future bias-related software engineering research, and for guiding educators in developing more state-of-the-art curricula.