Dhvani: A Weakly-supervised Phonemic Error Detection and Personalized Feedback System for Hindi

TL;DR

This paper introduces Dhvani, a novel CAPT system for Hindi that uses synthetic speech and personalized feedback to improve pronunciation, addressing a significant gap in Indian language learning tools.

Contribution

Dhvani is the first CAPT system for Hindi that combines synthetic mispronunciation generation with personalized feedback, leveraging Hindi's phonetic orthography.

Findings

Effective detection of mispronunciations in Hindi

Personalized feedback improves learner pronunciation

System demonstrates high accuracy in phonemic analysis

Abstract

Computer-Assisted Pronunciation Training (CAPT) has been extensively studied for English. However, there remains a critical gap in its application to Indian languages with a base of 1.5 billion speakers. Pronunciation tools tailored to Indian languages are strikingly lacking despite the fact that millions learn them every year. With over 600 million speakers and being the fourth most-spoken language worldwide, improving Hindi pronunciation is a vital first step toward addressing this gap. This paper proposes 1) Dhvani -- a novel CAPT system for Hindi, 2) synthetic speech generation for Hindi mispronunciations, and 3) a novel methodology for providing personalized feedback to learners. While the system often interacts with learners using Devanagari graphemes, its core analysis targets phonemic distinctions, leveraging Hindi's highly phonetic orthography to analyze mispronounced speech and provide targeted feedback.