NLP-Informed Dynamic Cognitive Diagnosis Modelling

TL;DR

This paper introduces a Bayesian framework that integrates natural language processing of question texts into dynamic cognitive diagnosis models to improve skill estimation in digital learning environments.

Contribution

It presents a novel method combining NLP-derived semantic information with Bayesian dynamic CDMs to enhance Q-matrix estimation and skill modeling.

Findings

Text-derived priors improve Q-matrix recovery.

The framework enhances model parameters in data-limited scenarios.

Application to digital reading data shows better skill development modeling.

Abstract

Digital learning platforms are increasingly used to support reading development while generating rich log files and item-level textual content. Using these data, this study proposes a dynamic cognitive diagnostic modelling (CDM) framework that incorporates text-derived semantic information to inform the estimation of the Q-matrix. We construct item-level semantic representations of question text and response options, and use these representations to define an informative prior on the Q-matrix. This approach treats text-derived signals as proxies for item complexity and cognitive demands, guiding the item-skill mapping in a data-driven manner. The proposed framework jointly estimates latent skill mastery profiles, item parameters, and transition dynamics over time within a Bayesian framework. We apply the model to data from Boost Reading, a digital reading supplement, focusing on students' vocabulary and comprehension skill development. We compare the proposed framework with a baseline model without any text information and show that the text-derived prior can improve Q-matrix recovery, particularly in settings where response data alone provide limited identification, as well as other model parameters for varying scenarios. This study provides a novel integration of natural language processing and dynamic CDMs, offering a data-driven approach to modelling skill acquisition and item-skill relationships in digital learning environments.