Evaluating Causal Discovery Algorithms for Path-Specific Fairness and Utility in Healthcare

TL;DR

This paper evaluates causal discovery algorithms in healthcare, focusing on their ability to recover structures and assess path-specific fairness, using expert-constructed benchmarks for synthetic and real clinical data.

Contribution

It introduces a benchmark framework with proxy ground-truth graphs for evaluating causal discovery algorithms in healthcare, emphasizing path-specific fairness and utility.

Findings

Peter-Clark best at structural recovery on synthetic data

Fast Causal Inference highest utility on heart failure data

Path-specific effects influence fairness-utility trade-offs

Abstract

Causal discovery in health data faces evaluation challenges when ground truth is unknown. We address this by collaborating with experts to construct proxy ground-truth graphs, establishing benchmarks for synthetic Alzheimer's disease and heart failure clinical records data. We evaluate the Peter-Clark, Greedy Equivalence Search, and Fast Causal Inference algorithms on structural recovery and path-specific fairness decomposition, going beyond composite fairness scores. On synthetic data, Peter-Clark achieved the best structural recovery. On heart failure data, Fast Causal Inference achieved the highest utility. For path-specific effects, ejection fraction contributed 3.37 percentage points to the indirect effect in the ground truth. These differences drove variations in the fairness-utility ratio across algorithms. Our results highlight the need for graph-aware fairness evaluation and fine-grained path-specific analysis when deploying causal discovery in clinical applications.