Candidate sentence selection for language learning exercises: from a comprehensive framework to an empirical evaluation

TL;DR

This paper introduces a comprehensive NLP-based framework for selecting candidate sentences for language learning exercises, emphasizing linguistic complexity and contextual dependence, validated through empirical evaluation with teachers and learners.

Contribution

It presents a novel hybrid system combining heuristics and machine learning for candidate sentence selection, addressing previously limited aspects in exercise generation.

Findings

System is useful for educational purposes

Empirical evaluation with teachers and learners supports effectiveness

Integrated into a free online learning platform

Abstract

We present a framework and its implementation relying on Natural Language Processing methods, which aims at the identification of exercise item candidates from corpora. The hybrid system combining heuristics and machine learning methods includes a number of relevant selection criteria. We focus on two fundamental aspects: linguistic complexity and the dependence of the extracted sentences on their original context. Previous work on exercise generation addressed these two criteria only to a limited extent, and a refined overall candidate sentence selection framework appears also to be lacking. In addition to a detailed description of the system, we present the results of an empirical evaluation conducted with language teachers and learners which indicate the usefulness of the system for educational purposes. We have integrated our system into a freely available online learning platform.