A Modest Pareto Optimisation Analysis of Dependency Parsers in 2021

TL;DR

This paper compares the accuracy and efficiency of three dependency parsers across diverse languages, highlighting the trade-offs and identifying biaffine parsing as a balanced default choice.

Contribution

It provides a Pareto front analysis of dependency parsers without pretrained models, offering insights into their efficiency-accuracy trade-offs.

Findings

Biaffine parsing is a well-balanced default.

Sequence-labelling parsing is faster for inference.

Pretrained models are excluded to focus on core efficiency.

Abstract

We evaluate three leading dependency parser systems from different paradigms on a small yet diverse subset of languages in terms of their accuracy-efficiency Pareto front. As we are interested in efficiency, we evaluate core parsers without pretrained language models (as these are typically huge networks and would constitute most of the compute time) or other augmentations that can be transversally applied to any of them. Biaffine parsing emerges as a well-balanced default choice, with sequence-labelling parsing being preferable if inference speed (but not training energy cost) is the priority.