Do Large Language Models Truly Grasp Mathematics? An Empirical Exploration From Cognitive Psychology

TL;DR

This study investigates whether large language models truly understand mathematics by testing their reasoning on modified cognitive reflection problems, revealing they rely more on pattern matching than genuine reasoning, thus challenging assumptions about their human-like cognition.

Contribution

The paper provides empirical evidence that mainstream LLMs do not exhibit human-like mathematical reasoning and primarily depend on pattern recognition, even with advanced prompting techniques.

Findings

LLMs perform poorly on modified CRT problems, with accuracy dropping by up to 50%.

LLMs rely mainly on pattern matching, akin to System 1 thinking.

Results challenge the notion that LLMs possess human-like mathematical reasoning.

Abstract

The cognitive mechanism by which Large Language Models (LLMs) solve mathematical problems remains a widely debated and unresolved issue. Currently, there is little interpretable experimental evidence that connects LLMs' problem-solving with human cognitive psychology.To determine if LLMs possess human-like mathematical reasoning, we modified the problems used in the human Cognitive Reflection Test (CRT). Our results show that, even with the use of Chains of Thought (CoT) prompts, mainstream LLMs, including the latest o1 model (noted for its reasoning capabilities), have a high error rate when solving these modified CRT problems. Specifically, the average accuracy rate dropped by up to 50% compared to the original questions.Further analysis of LLMs' incorrect answers suggests that they primarily rely on pattern matching from their training data, which aligns more with human intuition (System 1 thinking) rather than with human-like reasoning (System 2 thinking). This finding challenges the belief that LLMs have genuine mathematical reasoning abilities comparable to humans. As a result, this work may adjust overly optimistic views on LLMs' progress towards artificial general intelligence.