Selection-based Question Answering of an MOOC

TL;DR

This paper demonstrates how transformer-based deep learning models, specifically BERT, significantly improve real-time question answering in an MOOC discussion forum, reducing response times from minutes to seconds.

Contribution

It introduces a weighted similarity metric for matching questions and answers, enhancing automation in MOOC discussion forums using BERT over traditional word embeddings.

Findings

Response time reduced from 21 minutes to 0.3 seconds

Weighted similarity metric outperforms content-only methods

BERT-based models outperform Word2Vec in this context

Abstract

e-Yantra Robotics Competition (eYRC) is a unique Robotics Competition hosted by IIT Bombay that is actually an Embedded Systems and Robotics MOOC. Registrations have been growing exponentially in each year from 4500 in 2012 to over 34000 in 2019. In this 5-month long competition students learn complex skills under severe time pressure and have access to a discussion forum to post doubts about the learning material. Responding to questions in real-time is a challenge for project staff. Here, we illustrate the advantage of Deep Learning for real-time question answering in the eYRC discussion forum. We illustrate the advantage of Transformer based contextual embedding mechanisms such as Bidirectional Encoder Representation From Transformer (BERT) over word embedding mechanisms such as Word2Vec. We propose a weighted similarity metric as a measure of matching and find it more reliable than Content-Content or Title-Title similarities alone. The automation of replying to questions has brought the turn around response time(TART) down from a minimum of 21 mins to a minimum of 0.3 secs.