Dependency Parsing with LSTMs: An Empirical Evaluation

TL;DR

This paper presents an LSTM-based transition parser for dependency parsing, demonstrating improved accuracy especially on long-range dependencies, and highlights the importance of dropout regularization for better generalization.

Contribution

It extends previous neural parsers by incorporating LSTM units to model entire transition sequences, achieving superior performance on long-range dependencies.

Findings

LSTM parser is competitive with top feedforward parsers.

Over 3% improvement on long-range dependencies.

Dropout regularization enhances generalization.

Abstract

We propose a transition-based dependency parser using Recurrent Neural Networks with Long Short-Term Memory (LSTM) units. This extends the feedforward neural network parser of Chen and Manning (2014) and enables modelling of entire sequences of shift/reduce transition decisions. On the Google Web Treebank, our LSTM parser is competitive with the best feedforward parser on overall accuracy and notably achieves more than 3% improvement for long-range dependencies, which has proved difficult for previous transition-based parsers due to error propagation and limited context information. Our findings additionally suggest that dropout regularisation on the embedding layer is crucial to improve the LSTM's generalisation.