Neural Networks and Quantifier Conservativity: Does Data Distribution Affect Learnability?

TL;DR

This study investigates whether the distribution of quantifiers in training data influences neural networks' ability to learn conservative versus non-conservative determiners, suggesting that the observed bias in language acquisition may be innate or representational.

Contribution

The paper provides experimental evidence that the bias towards conservative quantifiers in language learning is not solely due to data distribution, implying innate or representational factors.

Findings

Neural networks struggle with non-conservative quantifiers regardless of data distribution.

Natural language data favors conservative quantifiers, but this does not explain learnability issues.

Bias in language acquisition may be innate or due to representational constraints.

Abstract

All known natural language determiners are conservative. Psycholinguistic experiments indicate that children exhibit a corresponding learnability bias when faced with the task of learning new determiners. However, recent work indicates that this bias towards conservativity is not observed during the training stage of artificial neural networks. In this work, we investigate whether the learnability bias exhibited by children is in part due to the distribution of quantifiers in natural language. We share results of five experiments, contrasted by the distribution of conservative vs. non-conservative determiners in the training data. We demonstrate that the aquisitional issues with non-conservative quantifiers can not be explained by the distribution of natural language data, which favors conservative quantifiers. This finding indicates that the bias in language acquisition data might be innate or representational.