Constraint Translation Candidates: A Bridge between Neural Query Translation and Cross-lingual Information Retrieval

TL;DR

This paper introduces a constrained translation approach that limits target vocabulary to important search index words, improving query translation quality and retrieval accuracy in cross-lingual information retrieval systems.

Contribution

It proposes a novel method to incorporate search index constraints into neural query translation, enhancing performance in real-world CLIR applications.

Findings

Improved translation quality over baseline NMT models.

Enhanced retrieval accuracy in the Aliexpress e-Commerce search system.

Effective use of constrained vocabulary at both training and inference stages.

Abstract

Query translation (QT) is a key component in cross-lingual information retrieval system (CLIR). With the help of deep learning, neural machine translation (NMT) has shown promising results on various tasks. However, NMT is generally trained with large-scale out-of-domain data rather than in-domain query translation pairs. Besides, the translation model lacks a mechanism at the inference time to guarantee the generated words to match the search index. The two shortages of QT result in readable texts for human but inadequate candidates for the downstream retrieval task. In this paper, we propose a novel approach to alleviate these problems by limiting the open target vocabulary search space of QT to a set of important words mined from search index database. The constraint translation candidates are employed at both of training and inference time, thus guiding the translation model to learn and generate well performing target queries. The proposed methods are exploited and examined in a real-word CLIR system--Aliexpress e-Commerce search engine. Experimental results demonstrate that our approach yields better performance on both translation quality and retrieval accuracy than the strong NMT baseline.