Flow-Adapter Architecture for Unsupervised Machine Translation

TL;DR

This paper introduces a novel flow-adapter architecture utilizing normalizing flows to model sentence-level latent representations for unsupervised machine translation, enabling independent training of languages and achieving competitive results.

Contribution

It is the first to apply normalizing flows and latent variables for unsupervised machine translation, improving modeling of language-specific representations.

Findings

Achieved competitive results on unsupervised MT benchmarks.

Enabled independent training of source and target languages.

Introduced a novel flow-based architecture for unsupervised translation.

Abstract

In this work, we propose a flow-adapter architecture for unsupervised NMT. It leverages normalizing flows to explicitly model the distributions of sentence-level latent representations, which are subsequently used in conjunction with the attention mechanism for the translation task. The primary novelties of our model are: (a) capturing language-specific sentence representations separately for each language using normalizing flows and (b) using a simple transformation of these latent representations for translating from one language to another. This architecture allows for unsupervised training of each language independently. While there is prior work on latent variables for supervised MT, to the best of our knowledge, this is the first work that uses latent variables and normalizing flows for unsupervised MT. We obtain competitive results on several unsupervised MT benchmarks.