Multi-class Hierarchical Question Classification for Multiple Choice Science Exams
Dongfang Xu, Peter Jansen, Jaycie Martin, Zhengnan Xie, Vikas Yadav,, Harish Tayyar Madabushi, Oyvind Tafjord, Peter Clark

TL;DR
This paper introduces the largest dataset for question classification in science exams, demonstrating that a BERT-based model significantly improves classification accuracy and enhances question answering systems.
Contribution
The creation of a large, fine-grained hierarchical question classification dataset and the development of a BERT-based model that outperforms previous methods.
Findings
BERT-based model achieves +0.12 MAP improvement.
State-of-the-art performance on open-domain and biomedical QC datasets.
Question classification improves QA accuracy by +1.7% P@1.
Abstract
Prior work has demonstrated that question classification (QC), recognizing the problem domain of a question, can help answer it more accurately. However, developing strong QC algorithms has been hindered by the limited size and complexity of annotated data available. To address this, we present the largest challenge dataset for QC, containing 7,787 science exam questions paired with detailed classification labels from a fine-grained hierarchical taxonomy of 406 problem domains. We then show that a BERT-based model trained on this dataset achieves a large (+0.12 MAP) gain compared with previous methods, while also achieving state-of-the-art performance on benchmark open-domain and biomedical QC datasets. Finally, we show that using this model's predictions of question topic significantly improves the accuracy of a question answering system by +1.7% P@1, with substantial future gains…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Educational Assessment and Pedagogy · Multimodal Machine Learning Applications
