Statistical Inference for Covariate-Adjusted and Interpretable Generalized Factor Model with Application to Testing Fairness

TL;DR

This paper develops a covariate-adjusted generalized factor model for large-scale assessment data, enabling valid inference on latent factors and covariate effects, with applications to fairness testing in educational assessments.

Contribution

It introduces a novel identifiability framework and estimation method for covariate effects in nonlinear latent factor models, addressing large sample challenges.

Findings

The proposed method achieves consistent and asymptotically normal estimates.

Numerical studies demonstrate the method's finite sample performance.

Application to PISA data illustrates practical utility in fairness testing.

Abstract

Latent variable models are popularly used to measure latent factors (e.g., abilities and personalities) from large-scale assessment data. Beyond understanding these latent factors, the covariate effect on responses controlling for latent factors is also of great scientific interest and has wide applications, such as evaluating the fairness of educational testing, where the covariate effect reflects whether a test question is biased toward certain individual characteristics (e.g., gender and race), taking into account their latent abilities. However, the large sample sizes and test lengths pose challenges to developing efficient methods and drawing valid inferences. Moreover, to accommodate the commonly encountered discrete responses, nonlinear latent factor models are often assumed, adding further complexity. To address these challenges, we consider a covariate-adjusted generalized factor model and develop novel and interpretable conditions to address the identifiability issue. Based on the identifiability conditions, we propose a joint maximum likelihood estimation method and establish estimation consistency and asymptotic normality results for the covariate effects. Furthermore, we derive estimation and inference results for latent factors and the factor loadings. We illustrate the finite sample performance of the proposed method through extensive numerical studies and an educational assessment dataset from the Programme for International Student Assessment (PISA).