NAVER LABS Europe Submission to the Instruction-following Track
Beomseok Lee, Marcely Zanon Boito, Laurent Besacier, Ioan Calapodescu

TL;DR
This paper presents NAVER LABS Europe's system for instruction-following speech processing, integrating pretrained speech and language models to perform ASR, speech translation, and question answering across multiple languages.
Contribution
It introduces a novel system combining pretrained speech and language models with instruction tuning for multilingual speech tasks.
Findings
Effective multilingual instruction-tuned speech processing system
Joint use of speech-to-LLM embedding projector and LoRA adapters
Achieved competitive performance in IWSLT 2025 challenge
Abstract
In this paper we describe NAVER LABS Europe submission to the instruction-following speech processing short track at IWSLT 2025. We participate in the constrained settings, developing systems that can simultaneously perform ASR, ST, and SQA tasks from English speech input into the following target languages: Chinese, Italian, and German. Our solution leverages two pretrained modules: (1) a speech-to-LLM embedding projector trained using representations from the SeamlessM4T-v2-large speech encoder; and (2) LoRA adapters trained on text data on top of a Llama-3.1-8B-Instruct. These modules are jointly loaded and further instruction-tuned for 1K steps on multilingual and multimodal data to form our final system submitted for evaluation.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsSpeech Recognition and Synthesis · Natural Language Processing Techniques · Speech and dialogue systems
