What's Going On in Neural Constituency Parsers? An Analysis

TL;DR

This paper investigates how modern neural constituency parsers implicitly learn information traditionally provided by explicit grammatical structures, showing neural models can encode similar linguistic knowledge.

Contribution

The study introduces a high-performance neural parser and analyzes its internal representations to demonstrate it captures information once supplied by explicit grammatical components.

Findings

Neural models encode information similar to grammars and lexicons.

Implicit learning reduces the need for explicit structural components.

The proposed model achieves 92.08 F1 on PTB.

Abstract

A number of differences have emerged between modern and classic approaches to constituency parsing in recent years, with structural components like grammars and feature-rich lexicons becoming less central while recurrent neural network representations rise in popularity. The goal of this work is to analyze the extent to which information provided directly by the model structure in classical systems is still being captured by neural methods. To this end, we propose a high-performance neural model (92.08 F1 on PTB) that is representative of recent work and perform a series of investigative experiments. We find that our model implicitly learns to encode much of the same information that was explicitly provided by grammars and lexicons in the past, indicating that this scaffolding can largely be subsumed by powerful general-purpose neural machinery.