Using Priming to Uncover the Organization of Syntactic Representations in Neural Language Models

TL;DR

This paper introduces a novel priming-based technique to analyze how neural language models organize syntactic information, revealing hierarchical and interpretable representations of complex sentence structures.

Contribution

It proposes a gradient similarity metric to reconstruct the syntactic representational space in neural models, providing new insights into their internal organization.

Findings

LSTM models' representations of relative clauses are hierarchically organized.

Models track abstract syntactic properties.

The technique reveals linguistically interpretable structures.

Abstract

Neural language models (LMs) perform well on tasks that require sensitivity to syntactic structure. Drawing on the syntactic priming paradigm from psycholinguistics, we propose a novel technique to analyze the representations that enable such success. By establishing a gradient similarity metric between structures, this technique allows us to reconstruct the organization of the LMs' syntactic representational space. We use this technique to demonstrate that LSTM LMs' representations of different types of sentences with relative clauses are organized hierarchically in a linguistically interpretable manner, suggesting that the LMs track abstract properties of the sentence.