Deep Biaffine Attention for Neural Dependency Parsing

TL;DR

This paper introduces a neural dependency parser using biaffine classifiers that achieves state-of-the-art results across multiple languages, outperforming previous graph-based methods and matching top transition-based parsers.

Contribution

The paper presents a novel biaffine attention mechanism for neural dependency parsing, significantly improving accuracy over prior graph-based approaches.

Findings

Achieved 95.7% UAS and 94.1% LAS on English PTB.

Outperformed previous graph-based parsers by 1.8% UAS.

Hyperparameter tuning greatly improved parsing accuracy.

Abstract

This paper builds off recent work from Kiperwasser & Goldberg (2016) using neural attention in a simple graph-based dependency parser. We use a larger but more thoroughly regularized parser than other recent BiLSTM-based approaches, with biaffine classifiers to predict arcs and labels. Our parser gets state of the art or near state of the art performance on standard treebanks for six different languages, achieving 95.7% UAS and 94.1% LAS on the most popular English PTB dataset. This makes it the highest-performing graph-based parser on this benchmark---outperforming Kiperwasser Goldberg (2016) by 1.8% and 2.2%---and comparable to the highest performing transition-based parser (Kuncoro et al., 2016), which achieves 95.8% UAS and 94.6% LAS. We also show which hyperparameter choices had a significant effect on parsing accuracy, allowing us to achieve large gains over other graph-based approaches.