Propensity Score Methods for Local Test Score Equating: Stratification and Inverse Probability Weighting

TL;DR

This paper introduces two new propensity score-based methods, stratification and inverse probability weighting, for local test score equating when anchor tests are unavailable, improving fairness in score comparisons.

Contribution

The paper develops and evaluates novel propensity score methods for local test equating without anchor tests, extending existing methodologies to covariate-only scenarios.

Findings

Both methods effectively adjust for group differences.

Performance depends on covariate-ability correlation strength.

Empirical and simulation results support their utility.

Abstract

In test equating, ensuring score comparability across different test forms is crucial but particularly challenging when test groups are non-equivalent and no anchor test is available. Local test equating aims to satisfy Lord's equity requirement by conditioning equating transformations on individual-level information, typically using anchor test scores as proxies for latent ability. However, anchor tests are not always available in practice. This paper introduces two novel propensity score-based methods for local equating: stratification and inverse probability weighting (IPW). These methods use covariates to account for group differences, with propensity scores serving as proxies for latent ability differences between test groups. The stratification method partitions examinees into comparable groups based on similar propensity scores, while IPW assigns weights inversely proportional to the probability of group membership. We evaluate these methods through empirical analysis and simulation studies. Results indicate both methods can effectively adjust for group differences, with their relative performance depending on the strength of covariate-ability correlations. The study extends local equating methodology to cases where only covariate information is available, providing testing programs with new tools for ensuring fair score comparability.