Datasets for Multilingual Answer Sentence Selection

TL;DR

This paper introduces high-quality multilingual answer sentence selection datasets for five European languages, created via supervised translation of English datasets, enabling better training of multilingual QA systems.

Contribution

It provides new multilingual AS2 datasets translated with LLMs, addressing resource scarcity and improving model performance across languages.

Findings

Datasets are high quality and effective for training multilingual AS2 models.

Multilingual models trained on these datasets outperform previous approaches.

The approach significantly reduces the performance gap between English and other languages.

Abstract

Answer Sentence Selection (AS2) is a critical task for designing effective retrieval-based Question Answering (QA) systems. Most advancements in AS2 focus on English due to the scarcity of annotated datasets for other languages. This lack of resources prevents the training of effective AS2 models in different languages, creating a performance gap between QA systems in English and other locales. In this paper, we introduce new high-quality datasets for AS2 in five European languages (French, German, Italian, Portuguese, and Spanish), obtained through supervised Automatic Machine Translation (AMT) of existing English AS2 datasets such as ASNQ, WikiQA, and TREC-QA using a Large Language Model (LLM). We evaluated our approach and the quality of the translated datasets through multiple experiments with different Transformer architectures. The results indicate that our datasets are pivotal in producing robust and powerful multilingual AS2 models, significantly contributing to closing the performance gap between English and other languages.