Cross-lingual Information Retrieval with BERT

TL;DR

This paper presents a deep relevance matching model based on multilingual BERT for cross-lingual information retrieval, demonstrating its effectiveness in retrieving Lithuanian documents with English queries and outperforming baseline methods.

Contribution

Introduces a novel BERT-based deep relevance matching model trained with weak supervision for cross-lingual IR tasks.

Findings

Model outperforms baseline approaches in Lithuanian document retrieval.

Effective use of weak supervision with parallel corpora for training.

Demonstrates BERT's capability in cross-lingual relevance modeling.

Abstract

Multiple neural language models have been developed recently, e.g., BERT and XLNet, and achieved impressive results in various NLP tasks including sentence classification, question answering and document ranking. In this paper, we explore the use of the popular bidirectional language model, BERT, to model and learn the relevance between English queries and foreign-language documents in the task of cross-lingual information retrieval. A deep relevance matching model based on BERT is introduced and trained by finetuning a pretrained multilingual BERT model with weak supervision, using home-made CLIR training data derived from parallel corpora. Experimental results of the retrieval of Lithuanian documents against short English queries show that our model is effective and outperforms the competitive baseline approaches.