Dimension Projection among Languages based on Pseudo-relevant Documents for Query Translation

TL;DR

This paper introduces a novel dictionary-based query translation method using dimension projection of embedded vectors from pseudo-relevant documents, significantly improving translation quality in cross-language information retrieval.

Contribution

It proposes a new approach for query translation that learns a transformation matrix between source and target language vectors based on pseudo-relevant documents.

Findings

Outperforms baseline word embedding methods.

Achieves up to 87% of machine translation performance.

Shows significant improvements in verbose queries.

Abstract

Using top-ranked documents in response to a query has been shown to be an effective approach to improve the quality of query translation in dictionary-based cross-language information retrieval. In this paper, we propose a new method for dictionary-based query translation based on dimension projection of embedded vectors from the pseudo-relevant documents in the source language to their equivalents in the target language. To this end, first we learn low-dimensional vectors of the words in the pseudo-relevant collections separately and then aim to find a query-dependent transformation matrix between the vectors of translation pairs appeared in the collections. At the next step, representation of each query term is projected to the target language and then, after using a softmax function, a query-dependent translation model is built. Finally, the model is used for query translation. Our experiments on four CLEF collections in French, Spanish, German, and Italian demonstrate that the proposed method outperforms a word embedding baseline based on bilingual shuffling and a further number of competitive baselines. The proposed method reaches up to 87% performance of machine translation (MT) in short queries and considerable improvements in verbose queries.