Qwen vs. Gemma Integration with Whisper: A Comparative Study in Multilingual SpeechLLM Systems

TL;DR

This paper compares the integration of Whisper with Gemma and Qwen models for multilingual speech recognition, demonstrating competitive results in the MLC-SLM Challenge 2025 through a multi-stage training approach.

Contribution

It introduces a novel system combining Whisper with Gemma and Qwen models, employing a three-stage training process for multilingual speech recognition.

Findings

Achieved 16.63% WER/CER with Gemma3-12B

Achieved 18.6% WER/CER with Qwen2.5-7B

Demonstrated competitive performance in MLC-SLM Challenge 2025

Abstract

This paper presents our system for the MLC-SLM Challenge 2025, focusing on multilingual speech recognition and language modeling with large language models (LLMs). Our approach combines a fine-tuned Whisper-large-v3 encoder with efficient projector architectures and various decoder configurations. We employ a three-stage training methodology that progressively optimizes the encoder, projector, and LLM components. Our system achieves competitive performance with a private test average WER/CER result of 16.63% using the Gemma3-12B and 18.6% using the Qwen2.5-7B as decoder-only language model.