Voxtral

TL;DR

Voxtral introduces two multimodal audio chat models that understand spoken audio and text, achieving state-of-the-art performance, with the smaller model capable of running locally and handling long audio and conversations.

Contribution

The paper presents Voxtral Mini and Small models with state-of-the-art audio understanding, long context handling, and new benchmarks for speech comprehension evaluation.

Findings

Voxtral models outperform existing models on audio benchmarks.

Voxtral Small can run locally due to its small size.

New benchmarks for speech understanding are introduced.

Abstract

We present Voxtral Mini and Voxtral Small, two multimodal audio chat models. Voxtral is trained to comprehend both spoken audio and text documents, achieving state-of-the-art performance across a diverse range of audio benchmarks, while preserving strong text capabilities. Voxtral Small outperforms a number of closed-source models, while being small enough to run locally. A 32K context window enables the model to handle audio files up to 40 minutes in duration and long multi-turn conversations. We also contribute three benchmarks for evaluating speech understanding models on knowledge and trivia. Both Voxtral models are released under Apache 2.0 license.