Zero-Shot Cross-Lingual Reranking with Large Language Models for Low-Resource Languages

TL;DR

This paper explores the effectiveness of large language models in zero-shot cross-lingual reranking for low-resource African languages, comparing multilingual and monolingual approaches using proprietary and open-source models.

Contribution

It provides the first comprehensive analysis of LLM-based reranking in African languages, highlighting the potential and limitations of cross-lingual capabilities in low-resource settings.

Findings

Reranking is most effective in English.

Cross-lingual reranking can be competitive with monolingual approaches.

Proprietary models outperform open-source models in this task.

Abstract

Large language models (LLMs) have shown impressive zero-shot capabilities in various document reranking tasks. Despite their successful implementations, there is still a gap in existing literature on their effectiveness in low-resource languages. To address this gap, we investigate how LLMs function as rerankers in cross-lingual information retrieval (CLIR) systems for African languages. Our implementation covers English and four African languages (Hausa, Somali, Swahili, and Yoruba) and we examine cross-lingual reranking with queries in English and passages in the African languages. Additionally, we analyze and compare the effectiveness of monolingual reranking using both query and document translations. We also evaluate the effectiveness of LLMs when leveraging their own generated translations. To get a grasp of the effectiveness of multiple LLMs, our study focuses on the proprietary models RankGPT-4 and RankGPT-3.5, along with the open-source model, RankZephyr. While reranking remains most effective in English, our results reveal that cross-lingual reranking may be competitive with reranking in African languages depending on the multilingual capability of the LLM.