Is MOOC Learning Different for Dropouts? A Visually-Driven, Multi-granularity Explanatory ML Approach

TL;DR

This paper introduces a multi-granularity, visually-driven explainable machine learning approach to analyze learner behavior in MOOCs, revealing distinct patterns between dropouts and completers through visualizations and statistical analysis.

Contribution

It proposes and compares different visualizations of clickstream data to explain learner dropout behavior, enhancing interpretability beyond traditional prediction models.

Findings

Non-completers tend to jump forward in sessions, often on a 'catch-up' path.

Completers exhibit more linear learning behavior.

Transition graphs reveal different activity patterns between dropouts and completers.

Abstract

Millions of people have enrolled and enrol (especially in the Covid-19 pandemic world) in MOOCs. However, the retention rate of learners is notoriously low. The majority of the research work on this issue focuses on predicting the dropout rate, but very few use explainable learning patterns as part of this analysis. However, visual representation of learning patterns could provide deeper insights into learners' behaviour across different courses, whilst numerical analyses can -- and arguably, should -- be used to confirm the latter. Thus, this paper proposes and compares different granularity visualisations for learning patterns (based on clickstream data) for both course completers and non-completers. In the large-scale MOOCs we analysed, across various domains, our fine-grained, fish-eye visualisation approach showed that non-completers are more likely to jump forward in their learning sessions, often on a 'catch-up' path, whilst completers exhibit linear behaviour. For coarser, bird-eye granularity visualisation, we observed learners' transition between types of learning activity, obtaining typed transition graphs. The results, backed up by statistical significance analysis and machine learning, provide insights for course instructors to maintain engagement of learners by adapting the course design to not just 'dry' predicted values, but explainable, visually viable paths extracted.