Automatically Suggesting Diverse Example Sentences for L2 Japanese Learners Using Pre-Trained Language Models

TL;DR

This paper explores using pre-trained language models to generate and evaluate diverse, proficiency-level-appropriate Japanese sentences for language learners, aiming to improve personalized learning tools.

Contribution

It introduces a novel system combining retrieval and zero-shot generation methods with PLMs for tailored sentence suggestions in L2 Japanese learning.

Findings

Retrieval approach was preferred for beginner and advanced levels

Participants showed disagreement on sentence quality ratings, except for difficulty

PLMs have potential to improve adaptive language learning systems

Abstract

Providing example sentences that are diverse and aligned with learners' proficiency levels is essential for fostering effective language acquisition. This study examines the use of Pre-trained Language Models (PLMs) to produce example sentences targeting L2 Japanese learners. We utilize PLMs in two ways: as quality scoring components in a retrieval system that draws from a newly curated corpus of Japanese sentences, and as direct sentence generators using zero-shot learning. We evaluate the quality of sentences by considering multiple aspects such as difficulty, diversity, and naturalness, with a panel of raters consisting of learners of Japanese, native speakers -- and GPT-4. Our findings suggest that there is inherent disagreement among participants on the ratings of sentence qualities, except for difficulty. Despite that, the retrieval approach was preferred by all evaluators, especially for beginner and advanced target proficiency, while the generative approaches received lower scores on average. Even so, our experiments highlight the potential for using PLMs to enhance the adaptability of sentence suggestion systems and therefore improve the language learning journey.