Unsupervised Multiple Choices Question Answering: Start Learning from Basic Knowledge

TL;DR

This paper explores an almost unsupervised approach to multiple choice question answering, leveraging noisy basic knowledge to guide model training, achieving competitive results without extensive labeled data.

Contribution

It introduces a novel unsupervised training method for MCQA models that uses basic knowledge and noisy signals, reducing reliance on labeled datasets.

Findings

Outperforms baseline methods on RACE dataset

Achieves comparable results to supervised approaches on MC500

Demonstrates effectiveness of noisy knowledge in guiding MCQA training

Abstract

In this paper, we study the possibility of almost unsupervised Multiple Choices Question Answering (MCQA). Starting from very basic knowledge, MCQA model knows that some choices have higher probabilities of being correct than the others. The information, though very noisy, guides the training of an MCQA model. The proposed method is shown to outperform the baseline approaches on RACE and even comparable with some supervised learning approaches on MC500.