Accent Normalization Using Self-Supervised Discrete Tokens with Non-Parallel Data

TL;DR

This paper introduces a self-supervised, non-parallel accent normalization method that converts accented speech into native-like speech while maintaining speaker identity, showing improved naturalness and accent reduction.

Contribution

It presents a novel pipeline using self-supervised discrete tokens and flow matching, advancing accent normalization without requiring parallel data.

Findings

Outperforms frame-to-frame baseline in naturalness

Reduces accentedness effectively across multiple English accents

Preserves speaker timbre during normalization

Abstract

Accent normalization converts foreign-accented speech into native-like speech while preserving speaker identity. We propose a novel pipeline using self-supervised discrete tokens and non-parallel training data. The system extracts tokens from source speech, converts them through a dedicated model, and synthesizes the output using flow matching. Our method demonstrates superior performance over a frame-to-frame baseline in naturalness, accentedness reduction, and timbre preservation across multiple English accents. Through token-level phonetic analysis, we validate the effectiveness of our token-based approach. We also develop two duration preservation methods, suitable for applications such as dubbing.