Testing Biased Randomization Assumptions and Quantifying Imperfect Matching and Residual Confounding in Matched Observational Studies

TL;DR

This paper introduces statistical tests to evaluate and quantify residual confounding in matched observational studies, emphasizing the importance of accounting for imperfect matching in causal inference.

Contribution

It develops two classes of exact tests for biased randomization assumptions and introduces the residual sensitivity value (RSV) to measure residual confounding.

Findings

The methodology quantifies residual confounding in matched samples.

Application to a clinical study demonstrates the practical utility.

Code implementation is provided for reproducibility.

Abstract

One central goal of design of observational studies is to embed non-experimental data into an approximate randomized controlled trial using statistical matching. Despite empirical researchers' best intention and effort to create high-quality matched samples, residual imbalance due to observed covariates not being well matched often persists. Although statistical tests have been developed to test the randomization assumption and its implications, few provide a means to quantify the level of residual confounding due to observed covariates not being well matched in matched samples. In this article, we develop two generic classes of exact statistical tests for a biased randomization assumption. One important by-product of our testing framework is a quantity called residual sensitivity value (RSV), which provides a means to quantify the level of residual confounding due to imperfect matching of observed covariates in a matched sample. We advocate taking into account RSV in the downstream primary analysis. The proposed methodology is illustrated by re-examining a famous observational study concerning the effect of right heart catheterization (RHC) in the initial care of critically ill patients. Code implementing the method can be found in the supplementary materials.