From Literature to Practice: Exploring Fairness Testing Tools for the Software Industry Adoption

TL;DR

This study evaluates existing fairness testing tools for AI in the software industry, highlighting usability issues and the need for more flexible, well-documented solutions to promote fairer AI systems.

Contribution

It provides an empirical assessment of current fairness testing tools, identifying key limitations and proposing design improvements for practical adoption in software development.

Findings

Many tools are difficult to use and lack documentation

Current tools focus on limited data types, reducing real-world applicability

Significant improvements are needed for tools to effectively support fairness testing

Abstract

In today's world, we need to ensure that AI systems are fair and unbiased. Our study looked at tools designed to test the fairness of software to see if they are practical and easy for software developers to use. We found that while some tools are cost-effective and compatible with various programming environments, many are hard to use and lack detailed instructions. They also tend to focus on specific types of data, which limits their usefulness in real-world situations. Overall, current fairness testing tools need significant improvements to better support software developers in creating fair and equitable technology. We suggest that new tools should be user-friendly, well-documented, and flexible enough to handle different kinds of data, helping developers identify and fix biases early in the development process. This will lead to more trustworthy and fair software for everyone.