Multi-lingual Common Semantic Space Construction via Cluster-consistent Word Embedding

TL;DR

This paper presents a novel method for constructing a multilingual semantic space by aligning words and clusters across languages using diverse signals and a cluster-consistent neural network, improving linguistic feature correlation and low-resource language tasks.

Contribution

It introduces a cluster-consistent correlational neural network for multilingual embedding, integrating multiple signals and linguistic knowledge for better cross-lingual semantic alignment.

Findings

Achieves higher correlation with linguistic features than existing methods.

Improves low-resource language name tagging F-score by up to 24.5%.

Outperforms state-of-the-art multi-lingual embedding techniques.

Abstract

We construct a multilingual common semantic space based on distributional semantics, where words from multiple languages are projected into a shared space to enable knowledge and resource transfer across languages. Beyond word alignment, we introduce multiple cluster-level alignments and enforce the word clusters to be consistently distributed across multiple languages. We exploit three signals for clustering: (1) neighbor words in the monolingual word embedding space; (2) character-level information; and (3) linguistic properties (e.g., apposition, locative suffix) derived from linguistic structure knowledge bases available for thousands of languages. We introduce a new cluster-consistent correlational neural network to construct the common semantic space by aligning words as well as clusters. Intrinsic evaluation on monolingual and multilingual QVEC tasks shows our approach achieves significantly higher correlation with linguistic features than state-of-the-art multi-lingual embedding learning methods do. Using low-resource language name tagging as a case study for extrinsic evaluation, our approach achieves up to 24.5\% absolute F-score gain over the state of the art.