# Unsupervised Training for Large Vocabulary Translation Using Sparse Lexicon and Word Classes

## TL;DR

This paper introduces an unsupervised method for large vocabulary translation that scales EM algorithm with sparsity and word class initialization, achieving promising results without parallel data.

## Contribution

It presents a novel scalable EM-based approach for unsupervised translation with large vocabularies, using sparsity and word classes for initialization.

## Key findings

- Successful scaling to hundreds of thousands of words
- Effective sparsity enforcement improves memory efficiency
- Promising results on large-scale unsupervised translation tasks

## Abstract

We address for the first time unsupervised training for a translation task with hundreds of thousands of vocabulary words. We scale up the expectation-maximization (EM) algorithm to learn a large translation table without any parallel text or seed lexicon. First, we solve the memory bottleneck and enforce the sparsity with a simple thresholding scheme for the lexicon. Second, we initialize the lexicon training with word classes, which efficiently boosts the performance. Our methods produced promising results on two large-scale unsupervised translation tasks.