Using Machine Bias To Measure Human Bias
Wanxue Dong, Maria De-Arteaga, Maytal Saar-Tsechansky

TL;DR
This paper introduces a machine learning framework to measure human decision biases in scenarios lacking clear correct labels, with theoretical and empirical validation, aiming to improve transparency and fairness in decision-making.
Contribution
The work presents a novel machine learning approach for bias measurement in human decisions without requiring gold standard labels, supported by theoretical guarantees and empirical evidence.
Findings
The proposed method outperforms existing bias detection techniques.
The framework provides theoretical guarantees for bias assessment.
Empirical results demonstrate improved bias measurement accuracy.
Abstract
Biased human decisions have consequential impacts across various domains, yielding unfair treatment of individuals and resulting in suboptimal outcomes for organizations and society. In recognition of this fact, organizations regularly design and deploy interventions aimed at mitigating these biases. However, measuring human decision biases remains an important but elusive task. Organizations are frequently concerned with mistaken decisions disproportionately affecting one group. In practice, however, this is typically not possible to assess due to the scarcity of a gold standard: a label that indicates what the correct decision would have been. In this work, we propose a machine learning-based framework to assess bias in human-generated decisions when gold standard labels are scarce. We provide theoretical guarantees and empirical evidence demonstrating the superiority of our method…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsOccupational Health and Safety Research
