Dependency Parsing with Bottom-up Hierarchical Pointer Networks

TL;DR

This paper introduces a bottom-up Hierarchical Pointer Network for dependency parsing, improving accuracy across multiple languages and surpassing previous models, including state-of-the-art results on English and Chinese datasets.

Contribution

It proposes a novel bottom-up Hierarchical Pointer Network and two new transition-based algorithms, enhancing dependency parsing performance.

Findings

Outperforms previous approaches on various languages

Achieves new state-of-the-art results on English and Chinese datasets

Effective with both non-contextualized and BERT-based embeddings

Abstract

Dependency parsing is a crucial step towards deep language understanding and, therefore, widely demanded by numerous Natural Language Processing applications. In particular, left-to-right and top-down transition-based algorithms that rely on Pointer Networks are among the most accurate approaches for performing dependency parsing. Additionally, it has been observed for the top-down algorithm that Pointer Networks' sequential decoding can be improved by implementing a hierarchical variant, more adequate to model dependency structures. Considering all this, we develop a bottom-up-oriented Hierarchical Pointer Network for the left-to-right parser and propose two novel transition-based alternatives: an approach that parses a sentence in right-to-left order and a variant that does it from the outside in. We empirically test the proposed neural architecture with the different algorithms on a wide variety of languages, outperforming the original approach in practically all of them and setting new state-of-the-art results on the English and Chinese Penn Treebanks for non-contextualized and BERT-based embeddings.