SC-Ques: A Sentence Completion Question Dataset for English as a Second Language Learners

TL;DR

This paper introduces a large-scale dataset of sentence completion questions for ESL learners, along with a benchmark using pre-trained language models to evaluate automatic solving capabilities.

Contribution

The paper provides the first extensive ESL sentence completion dataset, extsc{SC-Ques}, and establishes a benchmark for automatic question solving using large-scale pre-trained models.

Findings

Baseline models achieve moderate accuracy on the dataset.

Analysis reveals limitations and trade-offs in current models.

Dataset and code are publicly available for further research.

Abstract

Sentence completion (SC) questions present a sentence with one or more blanks that need to be filled in, three to five possible words or phrases as options. SC questions are widely used for students learning English as a Second Language (ESL). In this paper, we present a large-scale SC dataset, \textsc{SC-Ques}, which is made up of 289,148 ESL SC questions from real-world standardized English examinations. Furthermore, we build a comprehensive benchmark of automatically solving the SC questions by training the large-scale pre-trained language models on the proposed \textsc{SC-Ques} dataset. We conduct detailed analysis of the baseline models performance, limitations and trade-offs. The data and our code are available for research purposes from: \url{https://github.com/ai4ed/SC-Ques}.