A Theoretical Understanding of Chain-of-Thought: Coherent Reasoning and Error-Aware Demonstration

TL;DR

This paper provides a theoretical analysis of Chain-of-Thought prompting in large language models, showing that coherent reasoning improves error correction and prediction accuracy, and proposes an enhanced method incorporating both correct and incorrect reasoning paths.

Contribution

It offers a theoretical framework comparing stepwise and coherent CoT, revealing improved error correction and sensitivity analysis, and introduces a novel demonstration method for better reasoning.

Findings

Coherent CoT enhances transformer error correction capabilities.

Transformers are more sensitive to errors in intermediate reasoning steps.

Incorporating both correct and incorrect reasoning paths improves reasoning accuracy.

Abstract

Few-shot Chain-of-Thought (CoT) prompting has demonstrated strong performance in improving the reasoning capabilities of large language models (LLMs). While theoretical investigations have been conducted to understand CoT, the underlying transformer used in these studies isolates the CoT reasoning process into separated in-context learning steps (Stepwise ICL). In this work, we theoretically show that, compared to Stepwise ICL, the transformer gains better error correction ability and more accurate predictions if the reasoning from earlier steps (Coherent CoT) is integrated. Given that this coherent reasoning changes the behavior of the transformer, we further investigate the sensitivity of the transformer with Coherent CoT when the demonstration examples are corrupted at the inference stage. Our theoretical results indicate that the transformer is more sensitive to errors in intermediate reasoning steps than the final outcome. Building upon this observation, we propose an improvement on CoT by incorporating both correct and incorrect reasoning paths in the demonstration. Our experiments validate the effectiveness of the proposed approach.