Uncovering Bias Mechanisms in Observational Studies

TL;DR

This paper introduces a novel methodology that uses the relationship between bias magnitude and the predictive performance of nuisance function estimators to identify the underlying sources of bias in observational studies, enhancing causal inference.

Contribution

It presents a new approach for uncovering bias mechanisms in observational studies by analyzing the connection between bias and estimator performance, validated through experiments and a case study.

Findings

Effective in distinguishing bias sources such as hidden confounding and selection bias.

Validated on synthetic data and real-world case study.

Provides a new perspective for understanding bias in causal inference.

Abstract

Observational studies are a key resource for causal inference but are often affected by systematic biases. Prior work has focused mainly on detecting these biases, via sensitivity analyses and comparisons with randomized controlled trials, or mitigating them through debiasing techniques. However, there remains a lack of methodology for uncovering the underlying mechanisms driving these biases, e.g., whether due to hidden confounding or selection of participants. In this work, we show that the relationship between bias magnitude and the predictive performance of nuisance function estimators (in the observational study) can help distinguish among common sources of causal bias. We validate our methodology through extensive synthetic experiments and a real-world case study, demonstrating its effectiveness in revealing the mechanisms behind observed biases. Our framework offers a new lens for understanding and characterizing bias in observational studies, with practical implications for improving causal inference.