Cross-lingual Dependency Parsing with Unlabeled Auxiliary Languages

TL;DR

This paper introduces a method using adversarial training on unannotated auxiliary languages to learn language-agnostic representations, significantly improving cross-lingual dependency parsing across 28 languages.

Contribution

It proposes a novel adversarial training approach leveraging unlabeled data to enhance language-agnostic representations for cross-lingual transfer.

Findings

Adversarial training improves transfer performance across multiple languages.

The method is effective on 28 target languages.

Analysis confirms the learned representations are more language-invariant.

Abstract

Cross-lingual transfer learning has become an important weapon to battle the unavailability of annotated resources for low-resource languages. One of the fundamental techniques to transfer across languages is learning \emph{language-agnostic} representations, in the form of word embeddings or contextual encodings. In this work, we propose to leverage unannotated sentences from auxiliary languages to help learning language-agnostic representations. Specifically, we explore adversarial training for learning contextual encoders that produce invariant representations across languages to facilitate cross-lingual transfer. We conduct experiments on cross-lingual dependency parsing where we train a dependency parser on a source language and transfer it to a wide range of target languages. Experiments on 28 target languages demonstrate that adversarial training significantly improves the overall transfer performances under several different settings. We conduct a careful analysis to evaluate the language-agnostic representations resulted from adversarial training.