Partial-Mastery Cognitive Diagnosis Models

TL;DR

This paper introduces Partial-Mastery Cognitive Diagnosis Models (PM-CDMs), a novel approach that allows for partial mastery levels of attributes, improving model fit and diagnostic accuracy over traditional CDMs.

Contribution

The paper proposes PM-CDMs, a new class of models that generalize existing CDMs by incorporating partial mastery levels, with a Bayesian estimation approach and practical diagnostic tools.

Findings

PM-CDMs provide better model fit than traditional CDMs.

Partial mastery levels significantly impact diagnostic conclusions.

Simulation studies confirm accurate parameter recovery.

Abstract

Cognitive diagnosis models (CDMs) are a family of discrete latent attribute models that serve as statistical basis in educational and psychological cognitive diagnosis assessments. CDMs aim to achieve fine-grained inference on individuals' latent attributes, based on their observed responses to a set of designed diagnostic items. In the literature, CDMs usually assume that items require mastery of specific latent attributes and that each attribute is either fully mastered or not mastered by a given subject. We propose a new class of models, partial mastery CDMs (PM-CDMs), that generalizes CDMs by allowing for partial mastery levels for each attribute of interest. We demonstrate that PM-CDMs can be represented as restricted latent class models. Relying on the latent class representation, we propose a Bayesian approach for estimation. We present simulation studies to demonstrate parameter recovery, to investigate the impact of model misspecification with respect to partial mastery, and to develop diagnostic tools that could be used by practitioners to decide between CDMs and PM-CDMs. We use two examples of real test data -- the fraction subtraction and the English tests -- to demonstrate that employing PM-CDMs not only improves model fit, compared to CDMs, but also can make substantial difference in conclusions about attribute mastery. We conclude that PM-CDMs can lead to more effective remediation programs by providing detailed individual-level information about skills learned and skills that need to study.