Measuring covariate balance in weighted propensity score analyses by the weighted z-difference

TL;DR

This paper introduces the weighted z-difference as a new, simple measure for assessing covariate balance in propensity score weighted analyses, improving upon existing methods by providing a more reliable and versatile assessment tool.

Contribution

The paper proposes the weighted z-difference, a novel balance measure for propensity score analyses, demonstrated through simulations and a real cardiac surgery example.

Findings

Weighted z-difference is easy to compute for various covariate types.

It offers a more reliable assessment of covariate balance than standardized differences.

Q-Q plots facilitate comparison of balance across methods.

Abstract

Propensity score (PS) methods have been increasingly used in recent years when assessing treatment effects in nonrandomized studies. In terms of statistical methods, a number of new PS weighting methods were developed, and it was shown that they can outperform PS matching in efficiency of treatment effect estimation in different simulation settings. For assessing balance of covariates in treatment groups, PS weighting methods commonly use the weighted standardized difference, despite some deficiencies of this measure like, for example, the distribution of the weighted standardized difference depending on the sample size and on the distribution of weights. We introduce the weighted z-difference as a balance measure in PS weighting analyses and demonstrate its usage in a simulation study and by applying it to an example from cardiac surgery. The weighted z-difference is computationally simple and can be calculated for continuous, binary, ordinal and nominal covariates. By using Q-Q-plots we can compare the balance of PS weighted samples immediately to the balance in perfectly matched PS samples and to the expected balance in a randomized trial.