On the Eligibility of LLMs for Counterfactual Reasoning: A Decompositional Study

TL;DR

This paper introduces a decompositional framework to analyze how large language models perform counterfactual reasoning across various tasks and modalities, revealing key factors that influence their reasoning capabilities.

Contribution

It presents a structured approach to dissect counterfactual reasoning in LLMs, covering multiple tasks and modalities, and identifies factors affecting their reasoning performance.

Findings

LLMs struggle with counterfactual reasoning across tasks.

Modality type and intermediate reasoning significantly impact performance.

The framework aids in developing more reliable reasoning systems.

Abstract

Counterfactual reasoning has emerged as a crucial technique for generalizing the reasoning capabilities of large language models (LLMs). By generating and analyzing counterfactual scenarios, researchers can assess the adaptability and reliability of model decision-making. Although prior work has shown that LLMs often struggle with counterfactual reasoning, it remains unclear which factors most significantly impede their performance across different tasks and modalities. In this paper, we propose a decompositional strategy that breaks down the counterfactual generation from causality construction to the reasoning over counterfactual interventions. To support decompositional analysis, we investigate \ntask datasets spanning diverse tasks, including natural language understanding, mathematics, programming, and vision-language tasks. Through extensive evaluations, we characterize LLM behavior across each decompositional stage and identify how modality type and intermediate reasoning influence performance. By establishing a structured framework for analyzing counterfactual reasoning, this work contributes to the development of more reliable LLM-based reasoning systems and informs future elicitation strategies.