Child vs. machine language learning: Can the logical structure of human language unleash LLMs?

TL;DR

This paper explores how the logical structure of human language differs from current LLM training, suggesting that aligning models with human language logic could enhance their performance.

Contribution

It highlights the differences in learning biases between humans and LLMs and provides evidence that current models miss key logical aspects of language, proposing a new direction for improvement.

Findings

LLMs struggle with the logical structure of German plural formation

Current models do not fully capture language-inherent logic

Aligning models with human language structure may improve performance

Abstract

We argue that human language learning proceeds in a manner that is different in nature from current approaches to training LLMs, predicting a difference in learning biases. We then present evidence from German plural formation by LLMs that confirm our hypothesis that even very powerful implementations produce results that miss aspects of the logic inherent to language that humans have no problem with. We conclude that attention to the different structures of human language and artificial neural networks is likely to be an avenue to improve LLM performance.