On true versus estimated propensity scores for treatment effect estimation with discrete controls

TL;DR

This paper analyzes the variance of inverse propensity weighted estimators with discrete controls, showing estimated scores often outperform true scores, but true scores can be advantageous with prior knowledge.

Contribution

It derives finite sample variance expressions and compares the performance of true versus estimated propensity scores in treatment effect estimation.

Findings

Estimated propensity scores generally outperform true scores in finite samples.

Knowing the true propensity function can provide a statistical advantage.

Theoretical expressions support asymptotic theory on propensity score estimation.

Abstract

The finite sample variance of an inverse propensity weighted estimator is derived in the case of discrete control variables with finite support. The obtained expressions generally corroborate widely-cited asymptotic theory showing that estimated propensity scores are superior to true propensity scores in the context of inverse propensity weighting. However, similar analysis of a modified estimator demonstrates that foreknowledge of the true propensity function can confer a statistical advantage when estimating average treatment effects.