On the Challenges of Fully Incremental Neural Dependency Parsing

TL;DR

This paper investigates the feasibility of fully incremental neural dependency parsing using modern architectures, highlighting significant performance gaps compared to bidirectional methods and emphasizing the challenges of psycholinguistically plausible parsing.

Contribution

It introduces fully incremental neural dependency parsers with modern architectures and evaluates their performance, revealing substantial challenges and gaps compared to bidirectional approaches.

Findings

Fully incremental parsers lag behind bidirectional methods in performance.

Modern architectures face significant challenges in fully incremental parsing.

Psycholinguistically plausible parsing remains a complex challenge.

Abstract

Since the popularization of BiLSTMs and Transformer-based bidirectional encoders, state-of-the-art syntactic parsers have lacked incrementality, requiring access to the whole sentence and deviating from human language processing. This paper explores whether fully incremental dependency parsing with modern architectures can be competitive. We build parsers combining strictly left-to-right neural encoders with fully incremental sequence-labeling and transition-based decoders. The results show that fully incremental parsing with modern architectures considerably lags behind bidirectional parsing, noting the challenges of psycholinguistically plausible parsing.