FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoning

TL;DR

FACT-E introduces a causality-inspired evaluation framework that improves the reliability of assessing reasoning faithfulness in large language models by using controlled perturbations and combined metrics.

Contribution

It presents a novel causality-inspired method for evaluating and selecting trustworthy reasoning trajectories in LLMs, reducing bias and improving faithfulness assessment.

Findings

FACT-E enhances reasoning-trajectory selection in LLMs.

It provides more reliable detection of flawed reasoning under noisy conditions.

FACT-E improves the quality of in-context learning exemplars.

Abstract

Chain-of-Thought (CoT) prompting has improved LLM reasoning, but models often generate explanations that appear coherent while containing unfaithful intermediate steps. Existing self-evaluation approaches are prone to inherent biases: the model may confidently endorse coherence even when the step-to-step implication is not valid, leading to unreliable faithfulness evaluation. We propose FACT-E, a causality-inspired framework for evaluating CoT quality. FACT-E uses controlled perturbations as an instrumental signal to separate genuine step-to-step dependence from bias-driven artifacts, producing more reliable faithfulness estimates (\textit{intra-chain faithfulness}). To select trustworthy trajectories, FACT-E jointly considers \textit{intra-chain faithfulness} and \textit{CoT-to-answer consistency}, ensuring that selected chains are both faithful internally and supportive of the correct final answer. Experiments on GSM8K, MATH, and CommonsenseQA show that FACT-E improves reasoning-trajectory selection and yields stronger in-context learning exemplars. FACT-E also reliably detects flawed reasoning under noisy conditions, providing a robust metric for trustworthy LLM reasoning.