Discontinuous Grammar as a Foreign Language

TL;DR

This paper enhances sequence-to-sequence models for syntactic constituent parsing by enabling them to handle discontinuous structures, achieving state-of-the-art results on benchmarks and improving the integration of deep language understanding in AI systems.

Contribution

It introduces novel linearizations for sequence-to-sequence models to produce discontinuous syntactic structures, expanding their coverage and performance.

Findings

Achieved competitive results on discontinuous parsing benchmarks.

Obtained state-of-the-art scores on the English Penn Treebank.

Extended sequence-to-sequence models to handle complex syntactic phenomena.

Abstract

In order to achieve deep natural language understanding, syntactic constituent parsing is a vital step, highly demanded by many artificial intelligence systems to process both text and speech. One of the most recent proposals is the use of standard sequence-to-sequence models to perform constituent parsing as a machine translation task, instead of applying task-specific parsers. While they show a competitive performance, these text-to-parse transducers are still lagging behind classic techniques in terms of accuracy, coverage and speed. To close the gap, we here extend the framework of sequence-to-sequence models for constituent parsing, not only by providing a more powerful neural architecture for improving their performance, but also by enlarging their coverage to handle the most complex syntactic phenomena: discontinuous structures. To that end, we design several novel linearizations that can fully produce discontinuities and, for the first time, we test a sequence-to-sequence model on the main discontinuous benchmarks, obtaining competitive results on par with task-specific discontinuous constituent parsers and achieving state-of-the-art scores on the (discontinuous) English Penn Treebank.