Design of a novel Korean learning application for efficient pronunciation correction

TL;DR

This paper proposes a novel Korean language learning app focusing on pronunciation correction, utilizing speech recognition and Siamese networks to provide real-time feedback, addressing the lack of existing tools for foreigners.

Contribution

It introduces a new Korean pronunciation correction method using speech processing and neural networks, filling a gap in language learning applications for foreigners.

Findings

Designed a system with speech recognition and MFCC analysis

Proposed a Siamese network for pronunciation similarity scoring

Presented a novel correction method despite limited foreigner data

Abstract

The Korean wave, which denotes the global popularity of South Korea's cultural economy, contributes to the increasing demand for the Korean language. However, as there does not exist any application for foreigners to learn Korean, this paper suggested a design of a novel Korean learning application. Speech recognition, speech-to-text, and speech-to-waveform are the three key systems in the proposed system. The Google API and the librosa library will transform the user's voice into a sentence and MFCC. The software will then display the user's phrase and answer, with mispronounced elements highlighted in red, allowing users to more easily recognize the incorrect parts of their pronunciation. Furthermore, the Siamese network might utilize those translated spectrograms to provide a similarity score, which could subsequently be used to offer feedback to the user. Despite the fact that we were unable to collect sufficient foreigner data for this research, it is notable that we presented a novel Korean pronunciation correction method for foreigners.