Exploring Bayesian Deep Learning for Urgent Instructor Intervention Need in MOOC Forums

TL;DR

This paper investigates Bayesian deep learning methods to improve the detection of learners needing instructor intervention in MOOC forums, providing uncertainty estimates and enhancing trust in AI predictions.

Contribution

It introduces the application of Bayesian deep learning, specifically Monte Carlo Dropout and Variational Inference, to NLP tasks in MOOCs, offering a novel approach with better uncertainty quantification.

Findings

Bayesian methods provide valuable uncertainty measures.

Bayesian models achieve comparable or better accuracy.

Lower variance in predictions with Bayesian approaches.

Abstract

Massive Open Online Courses (MOOCs) have become a popular choice for e-learning thanks to their great flexibility. However, due to large numbers of learners and their diverse backgrounds, it is taxing to offer real-time support. Learners may post their feelings of confusion and struggle in the respective MOOC forums, but with the large volume of posts and high workloads for MOOC instructors, it is unlikely that the instructors can identify all learners requiring intervention. This problem has been studied as a Natural Language Processing (NLP) problem recently, and is known to be challenging, due to the imbalance of the data and the complex nature of the task. In this paper, we explore for the first time Bayesian deep learning on learner-based text posts with two methods: Monte Carlo Dropout and Variational Inference, as a new solution to assessing the need of instructor interventions for a learner's post. We compare models based on our proposed methods with probabilistic modelling to its baseline non-Bayesian models under similar circumstances, for different cases of applying prediction. The results suggest that Bayesian deep learning offers a critical uncertainty measure that is not supplied by traditional neural networks. This adds more explainability, trust and robustness to AI, which is crucial in education-based applications. Additionally, it can achieve similar or better performance compared to non-probabilistic neural networks, as well as grant lower variance.