Clustering Students and Inferring Skill Set Profiles with Skill Hierarchies

TL;DR

This paper explores optimal clustering methods for inferring student skill profiles from assessment data, especially considering skill hierarchies and incomplete profile populations, improving efficiency over traditional methods.

Contribution

It introduces a novel clustering approach combining hierarchical and k-means methods, tailored for skill hierarchies and partial profile presence, enhancing computational feasibility.

Findings

Empty k-means with new starting center method performs best.

Hierarchical clustering effectively models skill hierarchies.

Proposed methods outperform traditional approaches in accuracy and efficiency.

Abstract

Cognitive diagnosis models (CDMs) are a popular tool for assessing students' mastery of sets of skills. Given a set of $K$ skills tested on an assessment, students are classified into one of $2^K$ latent skill set profiles that represent whether they have mastered each skill or not. Traditional approaches to estimating these profiles are computationally intensive and become infeasible on large datasets. Instead, proxy skill estimates can be generated from the observed responses and then clustered, and these clusters can be assigned to different profiles. Building on previous work, we consider how to optimally perform this clustering when not all $2^K$ profiles are possible, e.g. because of hierarchical relationships among the skills, and when not all possible profiles are present in the population. We compare hierarchical clustering and several k-means variants, including semisupervised clustering using simulated student responses. The empty k-means algorithm paired with a novel method for generating starting centers yields the best overall performance.