Structured Training for Neural Network Transition-Based Parsing

TL;DR

This paper introduces a structured perceptron training approach for neural network transition-based dependency parsing, achieving state-of-the-art accuracy on the Penn Treebank by combining neural representations with structured learning.

Contribution

It presents a novel combination of neural network representations with structured perceptron training and beam-search decoding for dependency parsing.

Findings

Achieved 94.26% unlabeled attachment accuracy on Penn Treebank

Achieved 92.41% labeled attachment accuracy on Penn Treebank

Provided detailed analysis of model components contributing to accuracy

Abstract

We present structured perceptron training for neural network transition-based dependency parsing. We learn the neural network representation using a gold corpus augmented by a large number of automatically parsed sentences. Given this fixed network representation, we learn a final layer using the structured perceptron with beam-search decoding. On the Penn Treebank, our parser reaches 94.26% unlabeled and 92.41% labeled attachment accuracy, which to our knowledge is the best accuracy on Stanford Dependencies to date. We also provide in-depth ablative analysis to determine which aspects of our model provide the largest gains in accuracy.