Accent Recognition with Hybrid Phonetic Features

TL;DR

This paper presents a hybrid phonetic feature approach using auxiliary ASR tasks and combined acoustic model embeddings to improve accent recognition accuracy, achieving significant performance gains on the AESRC 2020 dataset.

Contribution

It introduces a novel hybrid structure that integrates fixed and trainable acoustic model embeddings for robust accent recognition using phonetic features.

Findings

6.57% relative improvement on validation set

7.28% relative improvement on test set

Enhanced robustness of accent recognition system

Abstract

The performance of voice-controlled systems is usually influenced by accented speech. To make these systems more robust, the frontend accent recognition (AR) technologies have received increased attention in recent years. As accent is a high-level abstract feature that has a profound relationship with the language knowledge, AR is more challenging than other language-agnostic audio classification tasks. In this paper, we use an auxiliary automatic speech recognition (ASR) task to extract language-related phonetic features. Furthermore, we propose a hybrid structure that incorporates the embeddings of both a fixed acoustic model and a trainable acoustic model, making the language-related acoustic feature more robust. We conduct several experiments on the Accented English Speech Recognition Challenge (AESRC) 2020 dataset. The results demonstrate that our approach can obtain a 6.57% relative improvement on the validation set. We also get a 7.28% relative improvement on the final test set for this competition, showing the merits of the proposed method.