Aequitas: A Bias and Fairness Audit Toolkit
Pedro Saleiro, Benedict Kuester, Loren Hinkson, Jesse London, Abby, Stevens, Ari Anisfeld, Kit T. Rodolfa, Rayid Ghani

TL;DR
Aequitas is an open-source toolkit designed to help developers and policymakers evaluate AI models for bias and fairness across various metrics and subgroups, promoting more equitable AI deployment.
Contribution
The paper introduces Aequitas, a comprehensive, easy-to-use toolkit that operationalizes multiple bias and fairness metrics for AI systems, filling a gap in practical bias auditing resources.
Findings
Enables testing of models for multiple bias metrics
Supports analysis across diverse population subgroups
Facilitates informed decision-making in AI deployment
Abstract
Recent work has raised concerns on the risk of unintended bias in AI systems being used nowadays that can affect individuals unfairly based on race, gender or religion, among other possible characteristics. While a lot of bias metrics and fairness definitions have been proposed in recent years, there is no consensus on which metric/definition should be used and there are very few available resources to operationalize them. Therefore, despite recent awareness, auditing for bias and fairness when developing and deploying AI systems is not yet a standard practice. We present Aequitas, an open source bias and fairness audit toolkit that is an intuitive and easy to use addition to the machine learning workflow, enabling users to seamlessly test models for several bias and fairness metrics in relation to multiple population sub-groups. Aequitas facilitates informed and equitable decisions…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEthics and Social Impacts of AI · Explainable Artificial Intelligence (XAI) · Adversarial Robustness in Machine Learning
