Predicting Abandonment in Online Coding Tutorials

TL;DR

This study explores machine learning techniques to predict when learners will abandon online coding tutorials, aiming to enable timely interventions to improve learner retention.

Contribution

It demonstrates the feasibility of predicting learner abandonment using interaction data and machine learning classifiers in an online programming game.

Findings

Classifiers predicted 61-76% of non-completers.

Features like help-seeking reduce abandonment risk.

Features indicating difficulty increase abandonment risk.

Abstract

Learners regularly abandon online coding tutorials when they get bored or frustrated, but there are few techniques for anticipating this abandonment to intervene. In this paper, we examine the feasibility of predicting abandonment with machine-learned classifiers. Using interaction logs from an online programming game, we extracted a collection of features that are potentially related to learner abandonment and engagement, then developed classifiers for each level. Across the first five levels of the game, our classifiers successfully predicted 61% to 76% of learners who did not complete the next level, achieving an average AUC of 0.68. In these classifiers, features negatively associated with abandonment included account activation and help-seeking behaviors, whereas features positively associated with abandonment included features indicating difficulty and disengagement. These findings highlight the feasibility of providing timely intervention to learners likely to quit.