Educational Question Mining At Scale: Prediction, Analysis and Personalization

TL;DR

This paper presents a scalable framework using Bayesian deep learning to analyze educational questions, quantify their quality and difficulty, and personalize question selection for students, improving online learning experiences.

Contribution

It introduces a novel application of p-VAE for question analysis, along with new metrics for quality and difficulty, enabling personalized question selection at scale.

Findings

Effective question quality and difficulty metrics developed

Framework shows high consistency with domain experts

Demonstrates promising results on large real-world dataset

Abstract

Online education platforms enable teachers to share a large number of educational resources such as questions to form exercises and quizzes for students. With large volumes of available questions, it is important to have an automated way to quantify their properties and intelligently select them for students, enabling effective and personalized learning experiences. In this work, we propose a framework for mining insights from educational questions at scale. We utilize the state-of-the-art Bayesian deep learning method, in particular partial variational auto-encoders (p-VAE), to analyze real students' answers to a large collection of questions. Based on p-VAE, we propose two novel metrics that quantify question quality and difficulty, respectively, and a personalized strategy to adaptively select questions for students. We apply our proposed framework to a real-world dataset with tens of thousands of questions and tens of millions of answers from an online education platform. Our framework not only demonstrates promising results in terms of statistical metrics but also obtains highly consistent results with domain experts' evaluation.