Machine Translation Impact in E-commerce Multilingual Search

TL;DR

This paper investigates how improvements in machine translation quality affect cross-lingual e-commerce search performance, identifying thresholds beyond which further translation quality gains do not significantly enhance retrieval accuracy.

Contribution

It systematically analyzes the relationship between MT quality and search performance across language pairs, providing strategies to optimize investment in MT improvements.

Findings

Identification of translation quality thresholds for effective retrieval

Development of techniques to compare query translations across languages

Guidelines for prioritizing language pairs for MT system enhancement

Abstract

Previous work suggests that performance of cross-lingual information retrieval correlates highly with the quality of Machine Translation. However, there may be a threshold beyond which improving query translation quality yields little or no benefit to further improve the retrieval performance. This threshold may depend upon multiple factors including the source and target languages, the existing MT system quality and the search pipeline. In order to identify the benefit of improving an MT system for a given search pipeline, we investigate the sensitivity of retrieval quality to the presence of different levels of MT quality using experimental datasets collected from actual traffic. We systematically improve the performance of our MT systems quality on language pairs as measured by MT evaluation metrics including Bleu and Chrf to determine their impact on search precision metrics and extract signals that help to guide the improvement strategies. Using this information we develop techniques to compare query translations for multiple language pairs and identify the most promising language pairs to invest and improve.