Practical Guide for Causal Pathways and Sub-group Disparity Analysis

TL;DR

This paper presents a causal disparity analysis framework that uncovers complex relationships and sub-group disparities related to sensitive attributes like race, aiding in bias quantification and fairness assessment in observational data.

Contribution

It introduces a novel causal decomposition and heterogeneity assessment methodology for analyzing disparities and biases in real-world datasets, especially focusing on sub-group impacts.

Findings

Sub-groups with the largest ML errors are most affected by disparities.

Causal analysis helps quantify biases and disentangle their effects on outcomes.

Grouping solely by sensitive attributes is insufficient for identifying impacted sub-groups.

Abstract

In this study, we introduce the application of causal disparity analysis to unveil intricate relationships and causal pathways between sensitive attributes and the targeted outcomes within real-world observational data. Our methodology involves employing causal decomposition analysis to quantify and examine the causal interplay between sensitive attributes and outcomes. We also emphasize the significance of integrating heterogeneity assessment in causal disparity analysis to gain deeper insights into the impact of sensitive attributes within specific sub-groups on outcomes. Our two-step investigation focuses on datasets where race serves as the sensitive attribute. The results on two datasets indicate the benefit of leveraging causal analysis and heterogeneity assessment not only for quantifying biases in the data but also for disentangling their influences on outcomes. We demonstrate that the sub-groups identified by our approach to be affected the most by disparities are the ones with the largest ML classification errors. We also show that grouping the data only based on a sensitive attribute is not enough, and through these analyses, we can find sub-groups that are directly affected by disparities. We hope that our findings will encourage the adoption of such methodologies in future ethical AI practices and bias audits, fostering a more equitable and fair technological landscape.