# CrossSpeech: Speaker-independent Acoustic Representation for Cross-lingual Speech Synthesis

## TL;DR

CrossSpeech introduces a novel approach to cross-lingual speech synthesis by disentangling speaker and language information in acoustic features, leading to improved speaker similarity and speech quality across languages.

## Contribution

It proposes a speaker-independent and speaker-dependent generator framework to better separate speaker and language features in cross-lingual TTS.

## Key findings

- Significant improvement in speaker similarity for cross-lingual TTS
- Effective disentanglement of speaker and language information
- Enhanced speech quality in cross-lingual synthesis

## Abstract

While recent text-to-speech (TTS) systems have made remarkable strides toward human-level quality, the performance of cross-lingual TTS lags behind that of intra-lingual TTS. This gap is mainly rooted from the speaker-language entanglement problem in cross-lingual TTS. In this paper, we propose CrossSpeech which improves the quality of cross-lingual speech by effectively disentangling speaker and language information in the level of acoustic feature space. Specifically, CrossSpeech decomposes the speech generation pipeline into the speaker-independent generator (SIG) and speaker-dependent generator (SDG). The SIG produces the speaker-independent acoustic representation which is not biased to specific speaker distributions. On the other hand, the SDG models speaker-dependent speech variation that characterizes speaker attributes. By handling each information separately, CrossSpeech can obtain disentangled speaker and language representations. From the experiments, we verify that CrossSpeech achieves significant improvements in cross-lingual TTS, especially in terms of speaker similarity to the target speaker.