False perspectives on human language: why statistics needs linguistics

TL;DR

This paper argues that statistical and structural linguistic models are not mutually exclusive, demonstrating that surprisal measures reflecting syntactic structure better explain language regularities.

Contribution

It shows empirically that surprisal models incorporating syntactic structure outperform purely surface-based models in capturing language regularities.

Findings

Syntactic surprisal models better predict language patterns

Statistical measures can be grounded in structural linguistic models

The dichotomy between statistics and linguistics is false

Abstract

A sharp tension exists about the nature of human language between two opposite parties: those who believe that statistical surface distributions, in particular using measures like surprisal, provide a better understanding of language processing, vs. those who believe that discrete hierarchical structures implementing linguistic information such as syntactic ones are a better tool. In this paper, we show that this dichotomy is a false one. Relying on the fact that statistical measures can be defined on the basis of either structural or non-structural models, we provide empirical evidence that only models of surprisal that reflect syntactic structure are able to account for language regularities.