Handling Students Dropouts in an LLM-driven Interactive Online Course Using Language Models

TL;DR

This study investigates dropout causes in AI-driven online courses, develops a predictive framework with high accuracy, and implements personalized interventions to improve student retention.

Contribution

It introduces a novel dropout prediction framework (CPADP) with up to 95.4% accuracy and a personalized re-engagement agent for AI-enhanced online courses.

Findings

Strong links between interaction patterns and dropout behaviors

CPADP achieves up to 95.4% prediction accuracy

Personalized email interventions effectively reduce dropouts

Abstract

Interactive online learning environments, represented by Massive AI-empowered Courses (MAIC), leverage LLM-driven multi-agent systems to transform passive MOOCs into dynamic, text-based platforms, enhancing interactivity through LLMs. This paper conducts an empirical study on a specific MAIC course to explore three research questions about dropouts in these interactive online courses: (1) What factors might lead to dropouts? (2) Can we predict dropouts? (3) Can we reduce dropouts? We analyze interaction logs to define dropouts and identify contributing factors. Our findings reveal strong links between dropout behaviors and textual interaction patterns. We then propose a course-progress-adaptive dropout prediction framework (CPADP) to predict dropouts with at most 95.4% accuracy. Based on this, we design a personalized email recall agent to re-engage at-risk students. Applied in the deployed MAIC system with over 3,000 students, the feasibility and effectiveness of our approach have been validated on students with diverse backgrounds.