Unsupervised and semi-supervised clustering methods to identify and refine participant experience levels in educational research

TL;DR

This paper evaluates unsupervised and semi-supervised clustering methods to classify educational participants' experience levels based on concept map data, revealing discrepancies with self-assessed experience and proposing metrics for characterisation.

Contribution

It introduces a novel application of clustering techniques to classify learner experience levels and compares these with self-assessments, enhancing understanding of conceptual development.

Findings

Disciplinary experience correlates with conceptual understanding.

Clustering identifies three distinct experience levels.

Node-level network metrics effectively characterize experience levels.

Abstract

The progression from novice to disciplinary expert is a longstanding area of inquiry in educational research. Studies investigating such progressions have often resorted to participants' self-assessments or other qualitative indicators as a starting point to define experience. But does a participant's estimated experience coincide with metrics derived from their conceptual understanding of a discipline? Using data extracted from over 150 concept maps, we first demonstrate that disciplinary experience is a reliable variable to explain differences in conceptual understanding across a highly diverse learners' population. Through a comparison of unsupervised and semi-supervised models, we then motivate clustering participants into three distinguished experience levels, and support such a classification performed in other studies of educational research. By analysing cluster composition, we also identify discrepancies between the perceived and predicted experience levels of the study participants. Lastly, for studies processing participants data through network analysis, we present insights into statistically significant metrics that can characterise each experience level, and advocate for the use of node-level metrics in such studies.