Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priors

TL;DR

This paper argues that LLMs operate through context-directed extrapolation from training priors, challenging extreme views of their capabilities and emphasizing predictable, controllable reasoning processes rather than emergent intelligence.

Contribution

It introduces the concept of context-directed extrapolation, clarifies LLM reasoning as predictable and controllable, and refocuses research on training data interactions rather than emergent agency.

Findings

LLMs extrapolate from training data priors guided by context.

Reasoning capabilities are predictable and controllable.

Fears of uncontrollable emergence of agency are unfounded.

Abstract

In this position paper we raise critical awareness of a realistic view of LLM capabilities that eschews extreme alternative views that LLMs are either 'stochastic parrots' or in possession of 'emergent' advanced reasoning capabilities, which, due to their unpredictable emergence, constitute an existential threat. Our middle-ground view is that LLMs extrapolate from priors from their training data while using context to guide the model to the appropriate priors; we call this "context-directed extrapolation." Specifically, this context direction is achieved through examples in base models, leading to in-context learning, while instruction tuning allows LLMs to perform similarly based on prompts rather than explicit examples. Under this view, substantiated though existing literature, while reasoning capabilities go well beyond stochastic parroting, such capabilities are predictable, controllable, not indicative of advanced reasoning akin to high-level cognitive capabilities in humans, and not infinitely scalable with additional training. As a result, fears of uncontrollable emergence of agency are allayed, while research advances are appropriately refocused on the processes of context-directed extrapolation and how this interacts with training data to produce valuable capabilities in LLMs. Future work can therefore explore alternative augmenting techniques that do not rely on inherent advanced reasoning in LLMs.