Multilingual Universal Sentence Encoder for Semantic Retrieval

TL;DR

This paper presents two multilingual sentence encoding models based on Transformer and CNN architectures, designed for semantic retrieval across 16 languages, achieving competitive performance with state-of-the-art models.

Contribution

Introduces two pre-trained multilingual sentence encoders using translation-based training, enabling effective semantic retrieval in multiple languages.

Findings

Competitive performance on semantic retrieval tasks

Effective cross-lingual transfer learning results

Models outperform some monolingual embeddings in English tasks

Abstract

We introduce two pre-trained retrieval focused multilingual sentence encoding models, respectively based on the Transformer and CNN model architectures. The models embed text from 16 languages into a single semantic space using a multi-task trained dual-encoder that learns tied representations using translation based bridge tasks (Chidambaram al., 2018). The models provide performance that is competitive with the state-of-the-art on: semantic retrieval (SR), translation pair bitext retrieval (BR) and retrieval question answering (ReQA). On English transfer learning tasks, our sentence-level embeddings approach, and in some cases exceed, the performance of monolingual, English only, sentence embedding models. Our models are made available for download on TensorFlow Hub.