Enriched In-Order Linearization for Faster Sequence-to-Sequence Constituent Parsing

TL;DR

This paper introduces an enriched in-order linearization method for sequence-to-sequence constituent parsing, achieving state-of-the-art accuracy and comparable speed to transition-based parsers on the English PTB dataset.

Contribution

It proposes an in-order linearization approach with enrichment, improving accuracy over top-down methods, and demonstrates that seq2seq models can match transition-based parsers in both speed and accuracy.

Findings

Achieved the best accuracy on the English PTB dataset for seq2seq parsers.

Matched the speed of state-of-the-art transition-based parsers.

Showed seq2seq models can be both accurate and fast.

Abstract

Sequence-to-sequence constituent parsing requires a linearization to represent trees as sequences. Top-down tree linearizations, which can be based on brackets or shift-reduce actions, have achieved the best accuracy to date. In this paper, we show that these results can be improved by using an in-order linearization instead. Based on this observation, we implement an enriched in-order shift-reduce linearization inspired by Vinyals et al. (2015)'s approach, achieving the best accuracy to date on the English PTB dataset among fully-supervised single-model sequence-to-sequence constituent parsers. Finally, we apply deterministic attention mechanisms to match the speed of state-of-the-art transition-based parsers, thus showing that sequence-to-sequence models can match them, not only in accuracy, but also in speed.