MENASpeechBank: A Reference Voice Bank with Persona-Conditioned Multi-Turn Conversations for AudioLLMs

TL;DR

This paper introduces MENASpeechBank, a diverse speech dataset from MENA speakers, and a synthetic data pipeline for creating persona-grounded multi-turn conversations to enhance AudioLLMs.

Contribution

It provides a new high-quality speech dataset and a controllable synthetic data generation pipeline for persona-based conversational audio modeling.

Findings

MENASpeechBank includes 18K utterances from 124 speakers across MENA.

Generated 417K role-play conversations for training and evaluation.

Synthetic data improves AudioLLMs' ability to handle persona and dialectal diversity.

Abstract

Audio large language models (AudioLLMs) enable instruction-following over speech and general audio, but progress is increasingly limited by the lack of diverse, conversational, instruction-aligned speech-text data. This bottleneck is especially acute for persona-grounded interactions and dialectal coverage, where collecting and releasing real multi-speaker recordings is costly and slow. We introduce MENASpeechBank, a reference speech bank comprising about 18K high-quality utterances from 124 speakers spanning multiple MENA countries, covering English, Modern Standard Arabic (MSA), and regional Arabic varieties. Building on this resource, we develop a controllable synthetic data pipeline that: (i) constructs persona profiles enriched with World Values Survey-inspired attributes, (ii) defines a taxonomy of about 5K conversational scenarios, (iii) matches personas to scenarios via semantic similarity, (iv) generates about 417K role-play conversations with an LLM where the user speaks as the persona and the assistant behaves as a helpful agent, and (v) synthesizes the user turns by conditioning on reference speaker audio to preserve speaker identity and diversity. We evaluate both synthetic and human-recorded conversations and provide detailed analysis. We will release MENASpeechBank and the generated conversations publicly for the community.