Simulated Reasoning is Reasoning

TL;DR

This paper explores how foundational models simulate reasoning through pattern imitation rather than human-like understanding, challenging traditional views and highlighting implications for AI safety and robustness.

Contribution

It argues that simulated reasoning in foundational models differs fundamentally from human reasoning, prompting a reevaluation of reasoning metaphors and safety considerations.

Findings

Foundational models reason by imitating thinking out loud and testing pathways.

Simulated reasoning lacks grounding and common sense, leading to brittleness.

The 'stochastic parrot' metaphor is outdated and should be abandoned.

Abstract

Reasoning has long been understood as a pathway between stages of understanding. Proper reasoning leads to understanding of a given subject. This reasoning was conceptualized as a process of understanding in a particular way, i.e., "symbolic reasoning". Foundational Models (FM) demonstrate that this is not a necessary condition for many reasoning tasks: they can "reason" by way of imitating the process of "thinking out loud", testing the produced pathways, and iterating on these pathways on their own. This leads to some form of reasoning that can solve problems on its own or with few-shot learning, but appears fundamentally different from human reasoning due to its lack of grounding and common sense, leading to brittleness of the reasoning process. These insights promise to substantially alter our assessment of reasoning and its necessary conditions, but also inform the approaches to safety and robust defences against this brittleness of FMs. This paper offers and discusses several philosophical interpretations of this phenomenon, argues that the previously apt metaphor of the "stochastic parrot" has lost its relevance and thus should be abandoned, and reflects on different normative elements in the safety- and appropriateness-considerations emerging from these reasoning models and their growing capacity.