FairytaleQA Translated: Enabling Educational Question and Answer Generation in Less-Resourced Languages

TL;DR

This paper introduces translated versions of the FairytaleQA dataset to support question answering and generation in less-resourced languages, using fine-tuned models and providing benchmarks and error analysis.

Contribution

It presents the first machine-translated FairytaleQA datasets for underrepresented languages, along with benchmarks and a case study for QA and QG tasks.

Findings

Established baseline benchmarks for QA and QG in translated datasets.

Analyzed error cases to guide future improvements.

Demonstrated feasibility of using modest models for multilingual narrative comprehension.

Abstract

Question Answering (QA) datasets are crucial in assessing reading comprehension skills for both machines and humans. While numerous datasets have been developed in English for this purpose, a noticeable void exists in less-resourced languages. To alleviate this gap, our paper introduces machine-translated versions of FairytaleQA, a renowned QA dataset designed to assess and enhance narrative comprehension skills in young children. By employing fine-tuned, modest-scale models, we establish benchmarks for both Question Generation (QG) and QA tasks within the translated datasets. In addition, we present a case study proposing a model for generating question-answer pairs, with an evaluation incorporating quality metrics such as question well-formedness, answerability, relevance, and children suitability. Our evaluation prioritizes quantifying and describing error cases, along with providing directions for future work. This paper contributes to the advancement of QA and QG research in less-resourced languages, promoting accessibility and inclusivity in the development of these models for reading comprehension. The code and data is publicly available at github.com/bernardoleite/fairytaleqa-translated.