Towards Accurate Phonetic Error Detection Through Phoneme Similarity Modeling

TL;DR

This paper introduces a phoneme recognition framework that models phoneme similarity to improve phonetic error detection, addressing variability from accents and dysfluencies, and establishes a new benchmark with a novel dataset and metrics.

Contribution

It proposes a novel phoneme similarity modeling approach with multi-task training for more accurate phonetic error detection and introduces a new dataset and evaluation metrics.

Findings

Improved phonetic error detection accuracy.

Open-sourced VCTK-accent dataset with phonetic errors.

New metrics for pronunciation difference assessment.

Abstract

Phonetic error detection, a core subtask of automatic pronunciation assessment, identifies pronunciation deviations at the phoneme level. Speech variability from accents and dysfluencies challenges accurate phoneme recognition, with current models failing to capture these discrepancies effectively. We propose a verbatim phoneme recognition framework using multi-task training with novel phoneme similarity modeling that transcribes what speakers actually say rather than what they're supposed to say. We develop and open-source \textit{VCTK-accent}, a simulated dataset containing phonetic errors, and propose two novel metrics for assessing pronunciation differences. Our work establishes a new benchmark for phonetic error detection.