MedMosaic: A Challenging Large Scale Benchmark of Diverse Medical Audio
Harshit Rajgarhia, Shuubham Ojha, Asif Shaik, Akhil Pothanapalli, Rachuri Lokesh, Abhishek Mukherji, Prasanna Desikan

TL;DR
MedMosaic is a comprehensive medical audio question-answering benchmark dataset designed to evaluate reasoning models in realistic clinical scenarios, highlighting current limitations and the need for advanced models.
Contribution
The paper introduces MedMosaic, a large-scale, diverse medical audio dataset with 46,701 QA pairs, to benchmark and analyze reasoning capabilities of multimodal models in medical contexts.
Findings
State-of-the-art models achieve only around 68.1% accuracy.
Reasoning remains challenging across all evaluated systems.
Performance varies significantly across different question types.
Abstract
We present MedMosaic, a medical audio question-answering dataset designed to benchmark language and audio reasoning models under realistic clinical constraints. Medical audio data is difficult to collect due to privacy regulations and high annotation costs arising from domain expertise. Thus, existing benchmarks tend to underrepresent complex medical audio scenarios. To address these challenges, MedMosaic features a diverse range of medical audio types, including condition-related physiological sounds, carefully constructed synthetic voices to mimic speech with artifacts as well as real short and long length clinical conversations to model varying context lengths. The dataset also features a total of 46,701 question-answer pairs, spanning categories such as multiple-choice, sequential multi-turn, and open-ended question-answers, enabling systematic evaluation of multi-hop reasoning and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
