Learning a Reversible Embedding Mapping using Bi-Directional Manifold Alignment

TL;DR

This paper introduces Bi-Directional Manifold Alignment (BDMA), a method for learning bijective, non-linear mappings between two manifolds, significantly reducing model count and enabling effective bidirectional language translation.

Contribution

The paper presents a novel BDMA approach that explicitly trains a bijective mapping between manifolds, reducing the number of models needed for language translation tasks.

Findings

BDMA reduces the number of models by 50%.

Models trained with BDMA perform well in reverse translation.

BDMA decreases overall model size.

Abstract

We propose a Bi-Directional Manifold Alignment (BDMA) that learns a non-linear mapping between two manifolds by explicitly training it to be bijective. We demonstrate BDMA by training a model for a pair of languages rather than individual, directed source and target combinations, reducing the number of models by 50%. We show that models trained with BDMA in the "forward" (source to target) direction can successfully map words in the "reverse" (target to source) direction, yielding equivalent (or better) performance to standard unidirectional translation models where the source and target language is flipped. We also show how BDMA reduces the overall size of the model.