Extraction of domain-specific bilingual lexicon from comparable corpora: compositional translation and ranking

TL;DR

This paper introduces a novel method for extracting morphologically complex bilingual lexicons from comparable corpora using compositional translation and ranking, achieving high accuracy in English-French and English-German pairs.

Contribution

It presents a new approach leveraging morpheme-level translation equivalences and ranking techniques to improve bilingual lexicon extraction from limited comparable corpora.

Findings

Average precision of 91% on Top1 candidate translation

Effective on English-French and English-German language pairs

Works with small specialized corpora (~400k words per language)

Abstract

This paper proposes a method for extracting translations of morphologically constructed terms from comparable corpora. The method is based on compositional translation and exploits translation equivalences at the morpheme-level, which allows for the generation of "fertile" translations (translation pairs in which the target term has more words than the source term). Ranking methods relying on corpus-based and translation-based features are used to select the best candidate translation. We obtain an average precision of 91% on the Top1 candidate translation. The method was tested on two language pairs (English-French and English-German) and with a small specialized comparable corpora (400k words per language).