Voxtral TTS

TL;DR

Voxtral TTS is a novel multilingual text-to-speech model that produces natural, expressive speech from minimal reference audio, utilizing a hybrid architecture and a custom speech tokenizer.

Contribution

It introduces a hybrid auto-regressive and flow-matching architecture with a new speech tokenizer trained from scratch, enabling high-quality multilingual voice cloning from limited data.

Findings

Voxtral TTS outperforms ElevenLabs Flash v2.5 in human evaluations for naturalness and expressivity.

The model achieves a 68.4% win rate in preference tests.

It can generate expressive speech from as little as 3 seconds of reference audio.

Abstract

We introduce Voxtral TTS, an expressive multilingual text-to-speech model that generates natural speech from as little as 3 seconds of reference audio. Voxtral TTS adopts a hybrid architecture that combines auto-regressive generation of semantic speech tokens with flow-matching for acoustic tokens. These tokens are encoded and decoded with Voxtral Codec, a speech tokenizer trained from scratch with a hybrid VQ-FSQ quantization scheme. In human evaluations conducted by native speakers, Voxtral TTS is preferred for multilingual voice cloning due to its naturalness and expressivity, achieving a 68.4\% win rate over ElevenLabs Flash v2.5. We release the model weights under a CC BY-NC license.