Stack-propagation: Improved Representation Learning for Syntax

TL;DR

This paper introduces stack-propagation, a method that improves syntax representation learning by using POS tags as regularizers, leading to more accurate dependency parsing across multiple languages without requiring POS tags at test time.

Contribution

The paper proposes a novel stack-propagation approach that leverages POS tags as regularizers in a stacked model pipeline, enhancing syntax learning for dependency parsing.

Findings

Achieved 1.3% higher accuracy than state-of-the-art graph-based models.

Improved accuracy by 2.7% over comparable greedy models.

Effective across 19 languages from Universal Dependencies.

Abstract

Traditional syntax models typically leverage part-of-speech (POS) information by constructing features from hand-tuned templates. We demonstrate that a better approach is to utilize POS tags as a regularizer of learned representations. We propose a simple method for learning a stacked pipeline of models which we call "stack-propagation". We apply this to dependency parsing and tagging, where we use the hidden layer of the tagger network as a representation of the input tokens for the parser. At test time, our parser does not require predicted POS tags. On 19 languages from the Universal Dependencies, our method is 1.3% (absolute) more accurate than a state-of-the-art graph-based approach and 2.7% more accurate than the most comparable greedy model.