On the statistical role of inexact matching in observational studies

TL;DR

This paper examines the limitations of inexact covariate matching in observational studies, showing it often leaves bias unaddressed and affects the validity of randomization tests, and suggests combining it with model-based adjustments.

Contribution

It demonstrates that inexact matching alone may not control bias or validate inference, and proposes combining it with model adjustments for improved robustness.

Findings

Inexact matching often leaves meaningful bias.

Standard randomization tests become asymptotically invalid with inexact matching.

Matching enhances robustness to model misspecification.

Abstract

In observational causal inference, exact covariate matching plays two statistical roles: (i) it effectively controls for bias due to measured confounding; (ii) it justifies assumption-free inference based on randomization tests. This paper shows that inexact covariate matching does not always play these same roles. We find that inexact matching often leaves behind statistically meaningful bias and that this bias renders standard randomization tests asymptotically invalid. We therefore recommend additional model-based covariate adjustment after inexact matching. In the framework of local misspecification, we prove that matching makes subsequent parametric analyses less sensitive to model selection or misspecification. We argue that gaining this robustness is the primary statistical role of inexact matching.