On the inconsistency of matching without replacement

TL;DR

This paper demonstrates that matching without replacement on propensity scores often leads to inconsistent estimators of the average treatment effect for the treated, unless specific assumptions are made about the distribution of propensity scores or confounding.

Contribution

It reveals the inconsistency issues of matching without replacement and clarifies conditions needed for consistent estimation, extending beyond propensity scores to other scoring methods.

Findings

Matching without replacement can be inconsistent for the average treatment effect.

Consistency requires assumptions about propensity score distribution or absence of confounding.

The issue is not specific to propensity scores but applies to other scoring methods as well.

Abstract

The paper shows that matching without replacement on propensity scores produces estimators that generally are inconsistent for the average treatment effect of the treated. To achieve consistency, practitioners must either assume that no units exist with propensity scores greater than one-half or assume that there is no confounding among such units. The result is not driven by the use of propensity scores, and similar artifacts arise when matching on other scores as long as it is without replacement.