LogiDynamics: Unraveling the Dynamics of Inductive, Abductive and Deductive Logical Inferences in LLM Reasoning

TL;DR

This paper systematically compares inductive, abductive, and deductive reasoning in large language models across modalities and task formats, revealing conditions where each approach excels and how advanced strategies can scale reasoning capabilities.

Contribution

It provides a comprehensive analysis of different logical inference mechanisms in LLMs and introduces strategies to enhance their reasoning performance.

Findings

System 2 inference generally outperforms System 1 in complex and visual/symbolic tasks.

Task format influences inference effectiveness, with free-text sometimes favoring System 1.

Advanced strategies like hypothesis selection improve System 2 reasoning scalability.

Abstract

Modern large language models (LLMs) employ diverse logical inference mechanisms for reasoning, making the strategic optimization of these approaches critical for advancing their capabilities. This paper systematically investigate the comparative dynamics of inductive (System 1) versus abductive/deductive (System 2) inference in LLMs. We utilize a controlled analogical reasoning environment, varying modality (textual, visual, symbolic), difficulty, and task format (MCQ / free-text). Our analysis reveals System 2 pipelines generally excel, particularly in visual/symbolic modalities and harder tasks, while System 1 is competitive for textual and easier problems. Crucially, task format significantly influences their relative advantage, with System 1 sometimes outperforming System 2 in free-text rule-execution. These core findings generalize to broader in-context learning. Furthermore, we demonstrate that advanced System 2 strategies like hypothesis selection and iterative refinement can substantially scale LLM reasoning. This study offers foundational insights and actionable guidelines for strategically deploying logical inference to enhance LLM reasoning. Resources are available at https://github.com/HKUST-KnowComp/LogiDynamics.