Unsupervised Cross-Lingual Transfer of Structured Predictors without Source Data

TL;DR

This paper introduces an unsupervised cross-lingual transfer method for structured prediction that effectively aggregates multiple models without requiring source data, improving label quality across 18 languages.

Contribution

It generalizes transfer methods to multiple input models and demonstrates that multiplying marginal probabilities yields better structures than union-based approaches.

Findings

Effective transfer across 18 languages.

Multiplying marginal probabilities improves label quality.

Less noisy labels for distant supervision.

Abstract

Providing technologies to communities or domains where training data is scarce or protected e.g., for privacy reasons, is becoming increasingly important. To that end, we generalise methods for unsupervised transfer from multiple input models for structured prediction. We show that the means of aggregating over the input models is critical, and that multiplying marginal probabilities of substructures to obtain high-probability structures for distant supervision is substantially better than taking the union of such structures over the input models, as done in prior work. Testing on 18 languages, we demonstrate that the method works in a cross-lingual setting, considering both dependency parsing and part-of-speech structured prediction problems. Our analyses show that the proposed method produces less noisy labels for the distant supervision.