Variable Selection in Latent Regression IRT Models via Knockoffs: An Application to International Large-scale Assessment in Education

TL;DR

This paper introduces a novel knockoff-based variable selection method for latent regression IRT models, applied to PISA data, effectively identifying non-cognitive factors linked to student performance while controlling false discovery rates.

Contribution

It develops a new knockoff approach tailored for latent variable models in ILSAs, addressing missing data and multiple comparisons in variable selection.

Findings

Successfully identified non-cognitive variables associated with science performance.

Controlled false discovery rate in variable selection.

Applied method to PISA data with promising results.

Abstract

International large-scale assessments (ILSAs) play an important role in educational research and policy making. They collect valuable data on education quality and performance development across many education systems, giving countries the opportunity to share techniques, organizational structures, and policies that have proven efficient and successful. To gain insights from ILSA data, we identify non-cognitive variables associated with students' academic performance. This problem has three analytical challenges: 1) academic performance is measured by cognitive items under a matrix sampling design; 2) there are many missing values in the non-cognitive variables; and 3) multiple comparisons due to a large number of non-cognitive variables. We consider an application to the Programme for International Student Assessment (PISA), aiming to identify non-cognitive variables associated with students' performance in science. We formulate it as a variable selection problem under a general latent variable model framework and further propose a knockoff method that conducts variable selection with a controlled error rate for false selections.