PTeacher: a Computer-Aided Personalized Pronunciation Training System with Exaggerated Audio-Visual Corrective Feedback

TL;DR

PTeacher is a personalized computer-aided pronunciation training system that uses exaggerated audio-visual feedback, tailored to individual learners' needs, to improve L2 English pronunciation effectively.

Contribution

This work introduces a novel personalized exaggeration mechanism for audio-visual feedback in pronunciation training, based on learner and teacher insights, enhancing effectiveness.

Findings

Significant improvement in learners' pronunciation accuracy.

Effective personalization of exaggeration levels based on learner profiles.

Enhanced learning efficiency demonstrated through user studies.

Abstract

Second language (L2) English learners often find it difficult to improve their pronunciations due to the lack of expressive and personalized corrective feedback. In this paper, we present Pronunciation Teacher (PTeacher), a Computer-Aided Pronunciation Training (CAPT) system that provides personalized exaggerated audio-visual corrective feedback for mispronunciations. Though the effectiveness of exaggerated feedback has been demonstrated, it is still unclear how to define the appropriate degrees of exaggeration when interacting with individual learners. To fill in this gap, we interview 100 L2 English learners and 22 professional native teachers to understand their needs and experiences. Three critical metrics are proposed for both learners and teachers to identify the best exaggeration levels in both audio and visual modalities. Additionally, we incorporate the personalized dynamic feedback mechanism given the English proficiency of learners. Based on the obtained insights, a comprehensive interactive pronunciation training course is designed to help L2 learners rectify mispronunciations in a more perceptible, understandable, and discriminative manner. Extensive user studies demonstrate that our system significantly promotes the learners' learning efficiency.