Assessing the Helpfulness of Learning Materials with Inference-Based Learner-Like Agent

TL;DR

This paper introduces an inference-based learner-like agent that assesses the helpfulness of learning materials for ESL learners by mimicking learner behavior, leading to improved learning outcomes and more effective sentence selection.

Contribution

The paper presents a novel inference-based agent that models learner behavior to evaluate and select helpful learning materials, outperforming prior methods in ESL sentence learning tasks.

Findings

The agent achieves the best performance in fill-in-the-blank and sentence selection tasks.

The agent helps students learn more easily and efficiently in classroom settings.

Over 17% of students improved their scores after using the agent.

Abstract

Many English-as-a-second language learners have trouble using near-synonym words (e.g., small vs.little; briefly vs.shortly) correctly, and often look for example sentences to learn how two nearly synonymous terms differ. Prior work uses hand-crafted scores to recommend sentences but has difficulty in adopting such scores to all the near-synonyms as near-synonyms differ in various ways. We notice that the helpfulness of the learning material would reflect on the learners' performance. Thus, we propose the inference-based learner-like agent to mimic learner behavior and identify good learning materials by examining the agent's performance. To enable the agent to behave like a learner, we leverage entailment modeling's capability of inferring answers from the provided materials. Experimental results show that the proposed agent is equipped with good learner-like behavior to achieve the best performance in both fill-in-the-blank (FITB) and good example sentence selection tasks. We further conduct a classroom user study with college ESL learners. The results of the user study show that the proposed agent can find out example sentences that help students learn more easily and efficiently. Compared to other models, the proposed agent improves the score of more than 17% of students after learning.