Unveiling Confirmation Bias in Chain-of-Thought Reasoning

TL;DR

This paper investigates confirmation bias in chain-of-thought reasoning of large language models, revealing how internal beliefs influence reasoning and answer prediction, and highlighting the need for strategies to mitigate bias for improved performance.

Contribution

It introduces a cognitive psychology perspective to analyze confirmation bias in LLMs' reasoning, providing empirical evidence and insights into how beliefs affect reasoning and answer prediction.

Findings

Confirmation bias affects reasoning generation and answer prediction in LLMs.

Model beliefs influence the utilization of rationales in reasoning tasks.

Task vulnerability to bias correlates with reasoning effectiveness across models.

Abstract

Chain-of-thought (CoT) prompting has been widely adopted to enhance the reasoning capabilities of large language models (LLMs). However, the effectiveness of CoT reasoning is inconsistent across tasks with different reasoning types. This work presents a novel perspective to understand CoT behavior through the lens of \textit{confirmation bias} in cognitive psychology. Specifically, we examine how model internal beliefs, approximated by direct question-answering probabilities, affect both reasoning generation ($Q \to R$) and reasoning-guided answer prediction ($QR \to A$) in CoT. By decomposing CoT into a two-stage process, we conduct a thorough correlation analysis in model beliefs, rationale attributes, and stage-wise performance. Our results provide strong evidence of confirmation bias in LLMs, such that model beliefs not only skew the reasoning process but also influence how rationales are utilized for answer prediction. Furthermore, the interplay between task vulnerability to confirmation bias and the strength of beliefs also provides explanations for CoT effectiveness across reasoning tasks and models. Overall, this study provides a valuable insight for the needs of better prompting strategies that mitigate confirmation bias to enhance reasoning performance. Code is available at \textit{https://github.com/yuewan2/biasedcot}.