A statistical framework for dynamic cognitive diagnosis in digital learning environments

TL;DR

This paper introduces a Bayesian dynamic cognitive diagnostic model that jointly estimates skill profiles and item mappings from log data, enhancing understanding of students' reading development over time.

Contribution

It presents a novel data-driven framework that estimates Q-matrices and skill transitions simultaneously, unlike traditional models requiring expert input.

Findings

Accurately recovers Q-matrices and skill profiles in simulations.

Effectively uncovers individual skill profiles in real data.

Demonstrates robustness across various sample sizes and Q-matrix sparsity.

Abstract

Reading is foundational for educational, employment, and economic outcomes, but a persistent proportion of students globally struggle to develop adequate reading skills. Some countries promote digital tools to support reading development, alongside regular classroom instruction. Such tools generate rich log data capturing students' behaviour and performance. This study proposes a dynamic cognitive diagnostic modeling (CDM) framework based on restricted latent class models to trace students' time-varying skills mastery using log files from digital tools. Unlike traditional CDMs that require expert-defined skill-item mappings (Q-matrix), our approach jointly estimates the Q-matrix and latent skill profiles, integrates log-derived covariates (e.g., reattempts, response times, counts of mastered items) and individual characteristics, and models transitions in mastery using a Bayesian estimation approach. Applied to real-world data, the model demonstrates practical value in educational settings by effectively uncovering individual skill profiles and the skill-item mappings. Simulation studies confirm robust recovery of Q-matrix structures and latent profiles with high accuracy under varied sample sizes, item counts and different sparsity of Q-matrices. The framework offers a data-driven, time-dependent restricted latent class modeling approach to understanding early reading development.