Question-type Identification for Academic Questions in Online Learning Platform

TL;DR

This paper presents a BERT-based ensemble model for accurately classifying twelve question types in online learning platforms, improving content understanding and student experience.

Contribution

It introduces a novel question-type classification system using weak supervision and manual annotation, with high accuracy for multiple question categories.

Findings

F1-score of 0.94 for MCQ binary classification

Effective multi-class classification for 12 question types

Model deployment enhances content understanding in online learning

Abstract

Online learning platforms provide learning materials and answers to students' academic questions by experts, peers, or systems. This paper explores question-type identification as a step in content understanding for an online learning platform. The aim of the question-type identifier is to categorize question types based on their structure and complexity, using the question text, subject, and structural features. We have defined twelve question-type classes, including Multiple-Choice Question (MCQ), essay, and others. We have compiled an internal dataset of students' questions and used a combination of weak-supervision techniques and manual annotation. We then trained a BERT-based ensemble model on this dataset and evaluated this model on a separate human-labeled test set. Our experiments yielded an F1-score of 0.94 for MCQ binary classification and promising results for 12-class multilabel classification. We deployed the model in our online learning platform as a crucial enabler for content understanding to enhance the student learning experience.