Substructure Distribution Projection for Zero-Shot Cross-Lingual Dependency Parsing

TL;DR

This paper introduces SubDP, a novel method for zero-shot cross-lingual dependency parsing that projects substructure distributions to improve parsing accuracy across multiple languages using minimal annotated data.

Contribution

SubDP is a new technique that projects substructure distributions separately for better zero-shot cross-lingual dependency parsing performance.

Findings

Outperforms prior methods on Universal Dependencies v2.2 test set

Achieves best labeled attachment scores on six out of eight languages

Enhances zero-shot parsing with very few supervised bitext pairs

Abstract

We present substructure distribution projection (SubDP), a technique that projects a distribution over structures in one domain to another, by projecting substructure distributions separately. Models for the target domains can be then trained, using the projected distributions as soft silver labels. We evaluate SubDP on zero-shot cross-lingual dependency parsing, taking dependency arcs as substructures: we project the predicted dependency arc distributions in the source language(s) to target language(s), and train a target language parser to fit the resulting distributions. When an English treebank is the only annotation that involves human effort, SubDP achieves better unlabeled attachment score than all prior work on the Universal Dependencies v2.2 (Nivre et al., 2020) test set across eight diverse target languages, as well as the best labeled attachment score on six out of eight languages. In addition, SubDP improves zero-shot cross-lingual dependency parsing with very few (e.g., 50) supervised bitext pairs, across a broader range of target languages.