Large Language Models and the Rationalist Empiricist Debate

TL;DR

This paper examines the debate between rationalism and empiricism in the context of large language models, arguing that LLMs do not provide evidence for or against human learning being empiricist, due to fundamental differences in learning processes.

Contribution

It clarifies the relevance of LLMs to the rationalist empiricist debate, emphasizing differences in learning mechanisms between humans and models and challenging claims of LLMs vindicating rationalism.

Findings

LLMs require rich stimuli, unlike humans who learn with limited input.

Externalized empiricism can accommodate innate biases if empirically determined.

Differences in learning processes suggest LLMs are not indicative of human learning modes.

Abstract

To many Chomsky's debates with Quine and Skinner are an updated version of the Rationalist Empiricist debates of the 17th century. The consensus being that Chomsky's Rationalism was victorious. This dispute has reemerged with the advent of Large Language Models. With some arguing that LLMs vindicate rationalism because of the necessity of building in innate biases to make them work. The necessity of building in innate biases is taken to prove that empiricism hasn't got the conceptual resources to explain linguistic competence. Such claims depend on the nature of the empiricism one is endorsing. Externalized Empiricism has no difficulties with innate apparatus once they are determined empirically (Quine 1969). Thus, externalized empiricism is not refuted because of the need to build in innate biases in LLMs. Furthermore, the relevance of LLMs to the rationalist empiricist debate in relation to humans is dubious. For any claim about whether LLMs learn in an empiricist manner to be relevant to humans it needs to be shown that LLMs and humans learn in the same way. Two key features distinguish humans and LLMs. Humans learn despite a poverty of stimulus and LLMs learn because of an incredibly rich stimulus. Human linguistic outputs are grounded in sensory experience and LLMs are not. These differences in how the two learn indicates that they both use different underlying competencies to produce their output. Therefore, any claims about whether LLMs learn in an empiricist manner are not relevant to whether humans learn in an empiricist manner.