Stress-Testing Causal Claims via Cardinality Repairs

TL;DR

This paper introduces SubCure, a framework for testing the robustness of causal claims by identifying minimal data modifications that significantly alter causal estimates, thereby exposing potential vulnerabilities in observational data analyses.

Contribution

The paper presents a novel cardinality repair approach for robustness auditing of causal claims, formalizes the NP-complete problem, and develops scalable algorithms using machine unlearning techniques.

Findings

SubCure uncovers small, impactful data subsets that alter causal conclusions.

It demonstrates effectiveness across diverse real-world datasets.

Traditional methods often miss these vulnerabilities.

Abstract

Causal analyses derived from observational data underpin high-stakes decisions in domains such as healthcare, public policy, and economics. Yet such conclusions can be surprisingly fragile: even minor data errors - duplicate records, or entry mistakes - may drastically alter causal relationships. This raises a fundamental question: how robust is a causal claim to small, targeted modifications in the data? Addressing this question is essential for ensuring the reliability, interpretability, and reproducibility of empirical findings. We introduce SubCure, a framework for robustness auditing via cardinality repairs. Given a causal query and a user-specified target range for the estimated effect, SubCure identifies a small set of tuples or subpopulations whose removal shifts the estimate into the desired range. This process not only quantifies the sensitivity of causal conclusions but also pinpoints the specific regions of the data that drive those conclusions. We formalize this problem under both tuple- and pattern-level deletion settings and show both are NP-complete. To scale to large datasets, we develop efficient algorithms that incorporate machine unlearning techniques to incrementally update causal estimates without retraining from scratch. We evaluate SubCure across four real-world datasets covering diverse application domains. In each case, it uncovers compact, high-impact subsets whose removal significantly shifts the causal conclusions, revealing vulnerabilities that traditional methods fail to detect. Our results demonstrate that cardinality repair is a powerful and general-purpose tool for stress-testing causal analyses and guarding against misleading claims rooted in ordinary data imperfections.