NMT-based Cross-lingual Document Embeddings

TL;DR

This paper proposes a constrained neural machine translation-based cross-lingual document embedding method that enhances embedding similarity without requiring translation during testing, improving efficiency and performance.

Contribution

It introduces a new constrained NV method that enforces embedding closeness for parallel documents, eliminating the need for translation at test time.

Findings

cNV performs as well as NV in classification tasks

cNV outperforms other methods that require decoding

The method is more lightweight and flexible

Abstract

This paper investigates a cross-lingual document embedding method that improves the current Neural machine Translation framework based Document Vector (NTDV or simply NV). NV is developed with a self-attention mechanism under the neural machine translation (NMT) framework. In NV, each pair of parallel documents in different languages are projected to the same shared layer in the model. However, the pair of NV embeddings are not guaranteed to be similar. This paper further adds a distance constraint to the training objective function of NV so that the two embeddings of a parallel document are required to be as close as possible. The new method will be called constrained NV (cNV). In a cross-lingual document classification task, the new cNV performs as well as NV and outperforms other published studies that require forward-pass decoding. Compared with the previous NV, cNV does not need a translator during testing, and so the method is lighter and more flexible.