Anveshana: A New Benchmark Dataset for Cross-Lingual Information Retrieval On English Queries and Sanskrit Documents

TL;DR

This paper introduces Anveshana, a benchmark dataset and evaluation framework for cross-lingual information retrieval between English queries and Sanskrit documents, utilizing advanced models and translation techniques to improve access to ancient texts.

Contribution

It provides a new publicly available dataset and evaluates multiple retrieval methods, including translation-based approaches, for Sanskrit-English cross-lingual retrieval.

Findings

DT methods outperform DR and QT in retrieval accuracy

Fine-tuned models effectively handle Sanskrit's linguistic features

The dataset enables further research in Sanskrit information retrieval

Abstract

The study presents a comprehensive benchmark for retrieving Sanskrit documents using English queries, focusing on the chapters of the Srimadbhagavatam. It employs a tripartite approach: Direct Retrieval (DR), Translation-based Retrieval (DT), and Query Translation (QT), utilizing shared embedding spaces and advanced translation methods to enhance retrieval systems in a RAG framework. The study fine-tunes state-of-the-art models for Sanskrit's linguistic nuances, evaluating models such as BM25, REPLUG, mDPR, ColBERT, Contriever, and GPT-2. It adapts summarization techniques for Sanskrit documents to improve QA processing. Evaluation shows DT methods outperform DR and QT in handling the cross-lingual challenges of ancient texts, improving accessibility and understanding. A dataset of 3,400 English-Sanskrit query-document pairs underpins the study, aiming to preserve Sanskrit scriptures and share their philosophical importance widely. Our dataset is publicly available at https://huggingface.co/datasets/manojbalaji1/anveshana