What Kind of Reasoning (if any) is an LLM actually doing? On the Stochastic Nature and Abductive Appearance of Large Language Models

TL;DR

This paper analyzes how LLMs generate plausible text through learned patterns, resembling abductive reasoning, but without genuine understanding or truth verification, impacting their evaluation and application.

Contribution

It clarifies that LLMs mimic abductive reasoning through stochastic pattern learning rather than performing actual reasoning processes.

Findings

LLMs produce plausible ideas without true understanding

Their outputs resemble abductive reasoning due to training on human texts

Critical assessment is necessary as LLMs cannot verify truth or explanations

Abstract

This article looks at how reasoning works in current Large Language Models (LLMs) that function using the token-completion method. It examines their stochastic nature and their similarity to human abductive reasoning. The argument is that these LLMs create text based on learned patterns rather than performing actual abductive reasoning. When their output seems abductive, this is largely because they are trained on human-generated texts that include reasoning structures. Examples are used to show how LLMs can produce plausible ideas, mimic commonsense reasoning, and give explanatory answers without being grounded in truth, semantics, verification, or understanding, and without performing any real abductive reasoning. This dual nature, where the models have a stochastic base but appear abductive in use, has important consequences for how LLMs are evaluated and applied. They can assist with generating ideas and supporting human thinking, but their outputs must be critically assessed because they cannot identify truth or verify their explanations. The article concludes by addressing five objections to these points, noting some limitations in the analysis, and offering an overall evaluation.