Zero-Shot Dependency Parsing with Worst-Case Aware Automated Curriculum Learning

TL;DR

This paper introduces a curriculum learning approach that dynamically optimizes zero-shot dependency parsing performance on low-resource and outlier languages, outperforming traditional sampling methods.

Contribution

It proposes a novel automated curriculum learning method tailored for zero-shot dependency parsing across diverse languages, especially low-resource and outlier languages.

Findings

Outperforms uniform sampling in zero-shot parsing

Significantly improves parsing accuracy on outlier languages

Demonstrates effectiveness of curriculum learning in multilingual models

Abstract

Large multilingual pretrained language models such as mBERT and XLM-RoBERTa have been found to be surprisingly effective for cross-lingual transfer of syntactic parsing models (Wu and Dredze 2019), but only between related languages. However, source and training languages are rarely related, when parsing truly low-resource languages. To close this gap, we adopt a method from multi-task learning, which relies on automated curriculum learning, to dynamically optimize for parsing performance on outlier languages. We show that this approach is significantly better than uniform and size-proportional sampling in the zero-shot setting.