M2QA: Multi-domain Multilingual Question Answering
Leon Engl\"ander, Hannah Sterz, Clifton Poth, Jonas Pfeiffer, Ilia, Kuznetsov, Iryna Gurevych

TL;DR
M2QA is a new benchmark for evaluating multilingual question answering across multiple domains, revealing significant challenges in transferability and robustness of current models to diverse language-domain combinations.
Contribution
The paper introduces M2QA, a comprehensive multilingual, multi-domain QA benchmark, and investigates the performance of models across languages and domains, highlighting transferability issues.
Findings
Performance varies significantly across domain-language pairs.
All models show notable drops when transferring to new language-domain combinations.
Current models are far from fully solving multilingual multi-domain QA.
Abstract
Generalization and robustness to input variation are core desiderata of machine learning research. Language varies along several axes, most importantly, language instance (e.g. French) and domain (e.g. news). While adapting NLP models to new languages within a single domain, or to new domains within a single language, is widely studied, research in joint adaptation is hampered by the lack of evaluation datasets. This prevents the transfer of NLP systems from well-resourced languages and domains to non-dominant language-domain combinations. To address this gap, we introduce M2QA, a multi-domain multilingual question answering benchmark. M2QA includes 13,500 SQuAD 2.0-style question-answer instances in German, Turkish, and Chinese for the domains of product reviews, news, and creative writing. We use M2QA to explore cross-lingual cross-domain performance of fine-tuned models and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Speech and dialogue systems · Natural Language Processing Techniques
