Transition-based Parsing with Lighter Feed-Forward Networks

TL;DR

This paper investigates how to create lighter, faster transition-based dependency parsers using feed-forward networks without sacrificing accuracy across various languages.

Contribution

It demonstrates that reducing feature complexity and embedding sizes can significantly speed up parsers while maintaining statistical equivalence.

Findings

Grand-daughter features can be removed without accuracy loss.

Embedding sizes can be substantially reduced.

Speed-ups are achieved with negligible impact on LAS.

Abstract

We explore whether it is possible to build lighter parsers, that are statistically equivalent to their corresponding standard version, for a wide set of languages showing different structures and morphologies. As testbed, we use the Universal Dependencies and transition-based dependency parsers trained on feed-forward networks. For these, most existing research assumes de facto standard embedded features and relies on pre-computation tricks to obtain speed-ups. We explore how these features and their size can be reduced and whether this translates into speed-ups with a negligible impact on accuracy. The experiments show that grand-daughter features can be removed for the majority of treebanks without a significant (negative or positive) LAS difference. They also show how the size of the embeddings can be notably reduced.