Finding Syntax in Human Encephalography with Beam Search

TL;DR

This paper demonstrates that recurrent neural network grammars combined with beam search can model human syntactic processing during language comprehension, as evidenced by electrophysiological response patterns.

Contribution

It introduces the use of RNNGs with beam search as a mechanistic model for human syntactic processing, validated by electrophysiological data.

Findings

RNNG with beam search produces early and late electrophysiological peaks similar to human responses.

Non-syntactic language models do not produce these electrophysiological effects.

Model comparisons suggest early peaks are due to syntactic composition in RNNGs.

Abstract

Recurrent neural network grammars (RNNGs) are generative models of (tree,string) pairs that rely on neural networks to evaluate derivational choices. Parsing with them using beam search yields a variety of incremental complexity metrics such as word surprisal and parser action count. When used as regressors against human electrophysiological responses to naturalistic text, they derive two amplitude effects: an early peak and a P600-like later peak. By contrast, a non-syntactic neural language model yields no reliable effects. Model comparisons attribute the early peak to syntactic composition within the RNNG. This pattern of results recommends the RNNG+beam search combination as a mechanistic model of the syntactic processing that occurs during normal human language comprehension.