NaijaRC: A Multi-choice Reading Comprehension Dataset for Nigerian Languages

TL;DR

This paper introduces NaijaRC, a multi-choice reading comprehension dataset for Nigerian languages, and evaluates baseline models and LLM prompting techniques to advance NLP resources for these languages.

Contribution

It creates the first high-school level reading comprehension dataset for Nigerian languages and benchmarks cross-lingual transfer and LLM prompting methods.

Findings

Cross-lingual transfer yields moderate accuracy improvements.

GPT-4 performs competitively with baseline models.

NaijaRC provides a valuable resource for NLP research in Nigerian languages.

Abstract

In this paper, we create NaijaRC: a new multi-choice Reading Comprehension dataset for three native Nigeria languages that is based on high-school reading comprehension examination. We provide baseline results by performing cross-lingual transfer using existing English RACE and Belebele training dataset based on a pre-trained encoder-only model. Additionally, we provide results by prompting large language models (LLMs) like GPT-4.