Auxiliary Task Guided Interactive Attention Model for Question Difficulty Prediction

TL;DR

This paper introduces a multi-task interactive attention model, Qdiff, that jointly predicts Bloom's Taxonomy and question difficulty, enhancing personalized assessments on online learning platforms.

Contribution

It proposes a novel multi-task learning approach with interactive attention to improve difficulty prediction by leveraging Bloom's Taxonomy information.

Findings

The method outperforms single-task difficulty prediction models.

Incorporating Bloom's Taxonomy improves difficulty prediction accuracy.

Soft labeling enables application to datasets lacking explicit Bloom's labels.

Abstract

Online learning platforms conduct exams to evaluate the learners in a monotonous way, where the questions in the database may be classified into Bloom's Taxonomy as varying levels in complexity from basic knowledge to advanced evaluation. The questions asked in these exams to all learners are very much static. It becomes important to ask new questions with different difficulty levels to each learner to provide a personalized learning experience. In this paper, we propose a multi-task method with an interactive attention mechanism, Qdiff, for jointly predicting Bloom's Taxonomy and difficulty levels of academic questions. We model the interaction between the predicted bloom taxonomy representations and the input representations using an attention mechanism to aid in difficulty prediction. The proposed learning method would help learn representations that capture the relationship between Bloom's taxonomy and difficulty labels. The proposed multi-task method learns a good input representation by leveraging the relationship between the related tasks and can be used in similar settings where the tasks are related. The results demonstrate that the proposed method performs better than training only on difficulty prediction. However, Bloom's labels may not always be given for some datasets. Hence we soft label another dataset with a model fine-tuned to predict Bloom's labels to demonstrate the applicability of our method to datasets with only difficulty labels.