AmQA: Amharic Question Answering Dataset

TL;DR

This paper introduces AmQA, the first publicly available Amharic question answering dataset, and provides baseline results to promote research in Amharic NLP.

Contribution

It creates the first Amharic QA dataset with 2628 question-answer pairs and evaluates baseline models to encourage further research.

Findings

Baseline F-score of 69.58 in reader-retriever QA

Baseline F-score of 71.74 in reading comprehension

Dataset to foster Amharic QA research

Abstract

Question Answering (QA) returns concise answers or answer lists from natural language text given a context document. Many resources go into curating QA datasets to advance robust models' development. There is a surge of QA datasets for languages like English, however, this is not true for Amharic. Amharic, the official language of Ethiopia, is the second most spoken Semitic language in the world. There is no published or publicly available Amharic QA dataset. Hence, to foster the research in Amharic QA, we present the first Amharic QA (AmQA) dataset. We crowdsourced 2628 question-answer pairs over 378 Wikipedia articles. Additionally, we run an XLMR Large-based baseline model to spark open-domain QA research interest. The best-performing baseline achieves an F-score of 69.58 and 71.74 in reader-retriever QA and reading comprehension settings respectively.