Causal Explanations for Disparate Trends: Where and Why?

TL;DR

ExDis is a framework that automatically identifies data regions and causal factors responsible for disparities between groups, aiding interpretable and actionable analysis of complex datasets.

Contribution

This paper introduces ExDis, a novel framework that discovers causal explanations for disparities, including data regions and factors, with an efficient algorithm and extensive experimental validation.

Findings

ExDis effectively identifies regions with significant disparities.

It uncovers causal factors contributing to disparities.

The method outperforms existing approaches in accuracy and scalability.

Abstract

During data analysis, we are often perplexed by certain disparities observed between two groups of interest within a dataset. To better understand an observed disparity, we need explanations that can pinpoint the data regions where the disparity is most pronounced, along with its causes, i.e., factors that alleviate or exacerbate the disparity. This task is complex and tedious, particularly for large and high-dimensional datasets, demanding an automatic system for discovering explanations (data regions and causes) of an observed disparity. It is critical that explanations for disparities are not only interpretable but also actionable-enabling users to make informed, data-driven decisions. This requires explanations to go beyond surface-level correlations and instead capture causal relationships. We introduce ExDis, a framework for discovering causal Explanations for Disparities between two groups of interest. ExDis identifies data regions (subpopulations) where disparities are most pronounced (or reversed), and associates specific factors that causally contribute to the disparity within each identified data region. We formally define the ExDis framework and the associated optimization problem, analyze its complexity, and develop an efficient algorithm to solve the problem. Through extensive experiments over three real-world datasets, we demonstrate that ExDis generates meaningful causal explanations, outperforms prior methods, and scales effectively to handle large, high-dimensional datasets.