Inaccessible Neural Language Models Could Reinvigorate Linguistic Nativism

TL;DR

This paper discusses how inaccessibility of large language models might promote nativist linguistic approaches and advocates for open sourcing LLMs to support diverse research methods.

Contribution

It highlights the risk of nativist bias due to inaccessible LLMs and calls for open sourcing to foster hybrid and empiricist approaches in linguistics research.

Findings

Inaccessible LLMs may lead to increased nativist bias.

Open sourcing LLMs can promote diverse linguistic research.

Encourages hybrid methods combining rules and deep learning.

Abstract

Large Language Models (LLMs) have been making big waves in the machine learning community within the past few years. The impressive scalability of LLMs due to the advent of deep learning can be seen as a continuation of empiricist lingusitic methods, as opposed to rule-based linguistic methods that are grounded in a nativist perspective. Current LLMs are generally inaccessible to resource-constrained researchers, due to a variety of factors including closed source code. This work argues that this lack of accessibility could instill a nativist bias in researchers new to computational linguistics, given that new researchers may only have rule-based, nativist approaches to study to produce new work. Also, given that there are numerous critics of deep learning claiming that LLMs and related methods may soon lose their relevancy, we speculate that such an event could trigger a new wave of nativism in the language processing community. To prevent such a dramatic shift and placing favor in hybrid methods of rules and deep learning, we call upon researchers to open source their LLM code wherever possible to allow both empircist and hybrid approaches to remain accessible.