Activation Steering for Accent-Neutralized Zero-Shot Text-To-Speech

TL;DR

This paper presents a training-free, inference-time method using activation steering to neutralize accents in zero-shot TTS, effectively reducing accent while maintaining speaker identity and generalizing to unseen speakers.

Contribution

Introduces a novel post-hoc activation steering technique for accent neutralization in zero-shot TTS without additional training.

Findings

Effectively reduces accent in generated speech

Preserves speaker timbre accurately

Generalizes well to unseen accented speakers

Abstract

Zero-shot Text-to-Speech (TTS) models can generate speech that captures both the voice timbre and accent of a reference speaker. However, disentangling these attributes remains challenging, as the output often inherits both the accent and timbre from the reference. In this study, we introduce a novel, post-hoc, and training-free approach to neutralize accent while preserving the speaker's original timbre, utilizing inference-time activation steering. We first extract layer-specific "steering vectors" offline, which are derived from the internal activation differences within the TTS model between accented and native speech. During inference, the steering vectors are applied to guide the model to produce accent-neutralized, timbre-preserving speech. Empirical results demonstrate that the proposed steering vectors effectively mitigate the output accent and exhibit strong generalizability to unseen accented speakers, offering a practical solution for accent-free voice cloning.