Comparative Analysis of Listwise Reranking with Large Language Models in Limited-Resource Language Contexts

TL;DR

This paper evaluates the effectiveness of large language models in listwise reranking for low-resource African languages, demonstrating their superiority over traditional methods in key ranking metrics.

Contribution

It provides a comparative analysis of proprietary LLMs for reranking in limited-resource languages, highlighting their potential and cost-effectiveness.

Findings

LLMs outperform BM25-DT in most metrics

RankGPT3.5 and RankClaude-sonnet show highest performance

LLMs are promising for low-resource language NLP tasks

Abstract

Large Language Models (LLMs) have demonstrated significant effectiveness across various NLP tasks, including text ranking. This study assesses the performance of large language models (LLMs) in listwise reranking for limited-resource African languages. We compare proprietary models RankGPT3.5, Rank4o-mini, RankGPTo1-mini and RankClaude-sonnet in cross-lingual contexts. Results indicate that these LLMs significantly outperform traditional baseline methods such as BM25-DT in most evaluation metrics, particularly in nDCG@10 and MRR@100. These findings highlight the potential of LLMs in enhancing reranking tasks for low-resource languages and offer insights into cost-effective solutions.