Causally Perturbed Fairness Testing
Chengwen Du, Tao Chen

TL;DR
CausalFT is a framework that uses causal inference to improve fairness testing in AI models by guiding perturbations based on causal relationships, significantly enhancing bug detection and bias resilience.
Contribution
It introduces a causal inference-based approach to guide perturbations in fairness testing, outperforming correlation-based methods and serving as a versatile framework adaptable to various generators.
Findings
CausalFT improves fairness bug detection in over 93% of cases.
It outperforms correlation-based feature ranking in 64% of cases.
CausalFT enhances bias resilience across nearly all tested scenarios.
Abstract
To mitigate unfair and unethical discrimination over sensitive features (e.g., gender, age, or race), fairness testing plays an integral role in engineering systems that leverage AI models to handle tabular data. A key challenge therein is how to effectively reveal fairness bugs under an intractable sample size using perturbation. Much current work has been focusing on designing the test sample generators, ignoring the valuable knowledge about data characteristics that can help guide the perturbation and hence limiting their full potential. In this paper, we seek to bridge such a gap by proposing a generic framework of causally perturbed fairness testing, dubbed CausalFT. Through causal inference, the key idea of CausalFT is to extract the most directly and causally relevant non-sensitive feature to its sensitive counterpart, which can jointly influence the prediction of the label. Such…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEthics and Social Impacts of AI · Adversarial Robustness in Machine Learning · Explainable Artificial Intelligence (XAI)
