Order-sensitive Neural Constituency Parsing

TL;DR

This paper introduces an order-sensitive decoding algorithm for neural constituency parsing that improves accuracy by considering span order, achieving better F1 scores while maintaining competitive decoding speed.

Contribution

It presents a novel order-sensitive decoding strategy that generalizes span-based parsing, incorporating span order into scoring for improved accuracy.

Findings

F1 score increased by 0.26% on Penn Treebank

F1 score increased by 0.35% on Chinese Treebank

Decoding speed comparable to state-of-the-art parsers

Abstract

We propose a novel algorithm that improves on the previous neural span-based CKY decoder for constituency parsing. In contrast to the traditional span-based decoding, where spans are combined only based on the sum of their scores, we introduce an order-sensitive strategy, where the span combination scores are more carefully derived from an order-sensitive basis. Our decoder can be regarded as a generalization over existing span-based decoder in determining a finer-grain scoring scheme for the combination of lower-level spans into higher-level spans, where we emphasize on the order of the lower-level spans and use order-sensitive span scores as well as order-sensitive combination grammar rule scores to enhance prediction accuracy. We implement the proposed decoding strategy harnessing GPU parallelism and achieve a decoding speed on par with state-of-the-art span-based parsers. Using the previous state-of-the-art model without additional data as our baseline, we outperform it and improve the F1 score on the Penn Treebank Dataset by 0.26% and on the Chinese Treebank Dataset by 0.35%.