CSCE: Boosting LLM Reasoning by Simultaneous Enhancing of Causal Significance and Consistency

TL;DR

This paper introduces CSCE, a novel non-chain reasoning framework that enhances large language models' reasoning by simultaneously improving causal significance and consistency, leading to better accuracy and efficiency.

Contribution

The paper proposes a new non-chain-based reasoning method that optimizes LLMs' causal understanding and consistency, moving away from traditional chain-of-thought approaches.

Findings

Improves reasoning success rate and speed.

Enhances LLMs' causal relationship understanding.

Demonstrates effectiveness of non-chain reasoning methods.

Abstract

Chain-based reasoning methods like chain of thought (CoT) play a rising role in solving reasoning tasks for large language models (LLMs). However, the causal hallucinations between a step of reasoning and corresponding state transitions are becoming a significant obstacle to advancing LLMs' reasoning capabilities, especially in long-range reasoning tasks. This paper proposes a non-chain-based reasoning framework for simultaneous consideration of causal significance and consistency, i.e., the Causal Significance and Consistency Enhancer (CSCE). We customize LLM's loss function utilizing treatment effect assessments to enhance its reasoning ability from two aspects: causal significance and consistency. This ensures that the model captures essential causal relationships and maintains robust and consistent performance across various scenarios. Additionally, we transform the reasoning process from the cascading multiple one-step reasoning commonly used in Chain-Based methods, like CoT, to a causal-enhanced method that outputs the entire reasoning process in one go, further improving the model's reasoning efficiency. Extensive experiments show that our method improves both the reasoning success rate and speed. These improvements further demonstrate that non-chain-based methods can also aid LLMs in completing reasoning tasks.