Recurrent Neural Network Grammars

TL;DR

This paper introduces recurrent neural network grammars, a probabilistic model combining phrase structure with neural networks, achieving superior parsing and language modeling performance in English and Chinese.

Contribution

The paper presents recurrent neural network grammars with efficient inference, improving upon previous models in parsing accuracy and language modeling performance.

Findings

Better parsing accuracy than previous supervised models in English

Superior language modeling than state-of-the-art RNNs in English and Chinese

Efficient inference procedures for practical application

Abstract

We introduce recurrent neural network grammars, probabilistic models of sentences with explicit phrase structure. We explain efficient inference procedures that allow application to both parsing and language modeling. Experiments show that they provide better parsing in English than any single previously published supervised generative model and better language modeling than state-of-the-art sequential RNNs in English and Chinese.