Learning from Partially Annotated Data: Example-aware Creation of Gap-filling Exercises for Language Learning

TL;DR

This paper introduces a neural network model for automatically generating French grammar gap-filling exercises from texts using only an example exercise, reducing the need for manual annotation.

Contribution

It presents a novel neural architecture and a real-world French dataset for exercise creation from partially annotated data, advancing automatic language learning tool development.

Findings

Model outperforms baseline classifier by 8% F1 score

Achieves an average F1 score of 82%

Provides publicly available dataset and implementation

Abstract

Since performing exercises (including, e.g., practice tests) forms a crucial component of learning, and creating such exercises requires non-trivial effort from the teacher, there is a great value in automatic exercise generation in digital tools in education. In this paper, we particularly focus on automatic creation of gapfilling exercises for language learning, specifically grammar exercises. Since providing any annotation in this domain requires human expert effort, we aim to avoid it entirely and explore the task of converting existing texts into new gap-filling exercises, purely based on an example exercise, without explicit instruction or detailed annotation of the intended grammar topics. We contribute (i) a novel neural network architecture specifically designed for aforementioned gap-filling exercise generation task, and (ii) a real-world benchmark dataset for French grammar. We show that our model for this French grammar gap-filling exercise generation outperforms a competitive baseline classifier by 8% in F1 percentage points, achieving an average F1 score of 82%. Our model implementation and the dataset are made publicly available to foster future research, thus offering a standardized evaluation and baseline solution of the proposed partially annotated data prediction task in grammar exercise creation.