Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training

TL;DR

This paper introduces a large-scale bilingual product description dataset and a novel cross-modal cross-lingual pre-training model to improve product-oriented machine translation, especially for e-commerce descriptions involving images.

Contribution

The paper presents a new dataset and a unified pre-training model that effectively align product images and bilingual texts for better translation quality.

Findings

The model outperforms state-of-the-art methods on multiple datasets.

Large-scale noisy data can be effectively utilized to enhance translation.

The dataset and code will be publicly released.

Abstract

Translating e-commercial product descriptions, a.k.a product-oriented machine translation (PMT), is essential to serve e-shoppers all over the world. However, due to the domain specialty, the PMT task is more challenging than traditional machine translation problems. Firstly, there are many specialized jargons in the product description, which are ambiguous to translate without the product image. Secondly, product descriptions are related to the image in more complicated ways than standard image descriptions, involving various visual aspects such as objects, shapes, colors or even subjective styles. Moreover, existing PMT datasets are small in scale to support the research. In this paper, we first construct a large-scale bilingual product description dataset called Fashion-MMT, which contains over 114k noisy and 40k manually cleaned description translations with multiple product images. To effectively learn semantic alignments among product images and bilingual texts in translation, we design a unified product-oriented cross-modal cross-lingual model (\upoc~) for pre-training and fine-tuning. Experiments on the Fashion-MMT and Multi30k datasets show that our model significantly outperforms the state-of-the-art models even pre-trained on the same dataset. It is also shown to benefit more from large-scale noisy data to improve the translation quality. We will release the dataset and codes at https://github.com/syuqings/Fashion-MMT.