A Multi-Level Trace Clustering Analysis Scheme for Measuring Students' Self-Regulated Learning Behavior in a Master-Based Online Learning Environment

TL;DR

This paper presents a multi-level clustering analysis scheme to examine students' self-regulated learning behaviors in an online mastery-based environment, revealing strategy shifts over time and informing instructional interventions.

Contribution

It introduces a novel three-level clustering analysis approach that effectively analyzes trace data with limited event types in online learning environments.

Findings

Most students start with productive strategies

Many students shift to less productive strategies over time

Distance metrics based on learning theory outperform other methods

Abstract

The study introduces a new analysis scheme to analyze trace data and visualize students' self-regulated learning strategies in a mastery-based online learning modules platform. The pedagogical design of the platform resulted in fewer event types and less variability in student trace data. The current analysis scheme overcomes those challenges by conducting three levels of clustering analysis. On the event level, mixture-model fitting is employed to distinguish between abnormally short and normal assessment attempts and study events. On the module level, trace level clustering is performed with three different methods for generating distance metrics between traces, with the best performing output used in the next step. On the sequence level, trace level clustering is performed on top of module-level clusters to reveal students' change of learning strategy over time. We demonstrated that distance metrics generated based on learning theory produced better clustering results than pure data-driven or hybrid methods. The analysis showed that most students started the semester with productive learning strategies, but a significant fraction shifted to a multitude of less productive strategies in response to increasing content difficulty and stress. The observations could prompt instructors to rethink conventional course structure and implement interventions to improve self-regulation at optimal times.