Deep Contextualized Self-training for Low Resource Dependency Parsing

TL;DR

This paper introduces Deep Contextualized Self-training (DCST), a semi-supervised method that improves low-resource dependency parsing by leveraging unlabeled data and representation models, outperforming existing self-training approaches.

Contribution

The paper presents a novel DCST algorithm that integrates representation models trained on sequence labeling tasks with a parser through gating, enhancing low-resource dependency parsing.

Findings

DCST outperforms traditional self-training methods.

DCST achieves significant improvements in low-resource and cross-domain scenarios.

The approach effectively leverages unlabeled data to reduce annotation costs.

Abstract

Neural dependency parsing has proven very effective, achieving state-of-the-art results on numerous domains and languages. Unfortunately, it requires large amounts of labeled data, that is costly and laborious to create. In this paper we propose a self-training algorithm that alleviates this annotation bottleneck by training a parser on its own output. Our Deep Contextualized Self-training (DCST) algorithm utilizes representation models trained on sequence labeling tasks that are derived from the parser's output when applied to unlabeled data, and integrates these models with the base parser through a gating mechanism. We conduct experiments across multiple languages, both in low resource in-domain and in cross-domain setups, and demonstrate that DCST substantially outperforms traditional self-training as well as recent semi-supervised training methods.