Likely to stop? Predicting Stopout in Massive Open Online Courses

TL;DR

This paper presents a scalable methodology for predicting student stopout in MOOCs using machine learning, achieving high accuracy with models trained on extensive feature engineering and data analysis.

Contribution

It introduces a comprehensive, end-to-end predictive framework for MOOC stopout, utilizing over 25 features and analyzing thousands of models to demonstrate prediction feasibility.

Findings

Models achieved up to 0.95 AUC for one-week-ahead predictions.

Prediction accuracy remains reasonable (AUC 0.7) even with minimal data at course end.

Stopout prediction is a tractable problem with effective machine learning approaches.

Abstract

Understanding why students stopout will help in understanding how students learn in MOOCs. In this report, part of a 3 unit compendium, we describe how we build accurate predictive models of MOOC student stopout. We document a scalable, stopout prediction methodology, end to end, from raw source data to model analysis. We attempted to predict stopout for the Fall 2012 offering of 6.002x. This involved the meticulous and crowd-sourced engineering of over 25 predictive features extracted for thousands of students, the creation of temporal and non-temporal data representations for use in predictive modeling, the derivation of over 10 thousand models with a variety of state-of-the-art machine learning techniques and the analysis of feature importance by examining over 70000 models. We found that stop out prediction is a tractable problem. Our models achieved an AUC (receiver operating characteristic area-under-the-curve) as high as 0.95 (and generally 0.88) when predicting one week in advance. Even with more difficult prediction problems, such as predicting stop out at the end of the course with only one weeks' data, the models attained AUCs of 0.7.