The Role of Deductive and Inductive Reasoning in Large Language Models
Chengkun Cai, Xu Zhao, Haoliang Liu, Zhongyu Jiang, Tianfang Zhang, Zongkai Wu, Jenq-Neng Hwang, Lei Li
TL;DR
This paper introduces the DID framework that enhances large language model reasoning by dynamically integrating deductive and inductive approaches, guided by cognitive science principles, leading to improved accuracy and efficiency.
Contribution
The paper presents a novel dual-metric evaluation system and a dynamic reasoning strategy that significantly improves LLM reasoning performance across multiple benchmarks.
Findings
Achieved 70.3% accuracy on AIW benchmark, outperforming Tree of Thought.
Demonstrated lower computational costs compared to traditional methods.
Enhanced reasoning quality and adaptability in complex scenarios.
Abstract
Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning tasks, yet their reliance on static prompt structures and limited adaptability to complex scenarios remains a significant challenge. In this paper, we propose the Deductive and InDuctive(DID) method, a novel framework that enhances LLM reasoning by dynamically integrating both deductive and inductive reasoning approaches. Drawing from cognitive science principles, DID implements a dual-metric complexity evaluation system that combines Littlestone dimension and information entropy to precisely assess task difficulty and guide decomposition strategies. DID enables the model to progressively adapt its reasoning pathways based on problem complexity, mirroring human cognitive processes. We evaluate DID's effectiveness across multiple benchmarks, including the AIW and MR-GSM8K, as well as our custom Holiday…
Peer Reviews
Decision·ICLR 2025 Conference Withdrawn Submission
Originality: The DID framework’s combination of inductive and deductive reasoning is an interesting approach to enhancing reasoning flexibility.
Clarity of Figures and Descriptions: Some figures (e.g. 1, 4) lack essential information, such as dataset details, metric explanations, and setup descriptions, making it difficult to interpret results. Prompt Strategy Details: As a prompt strategy paper, the DID method should explicitly define its prompt strategy. Currently, the methodology is insufficiently explained, with few details about how the DID prompts are structured or adapted dynamically. Reproducibility: The paper lacks reproducibi
1. Flexible approach: While the approach is not entirely novel, it does appear more flexible than existing methods. 2. Improved accuracy: The authors demonstrate emperically that DID method somewhat outperforms the existing CoT methods.
1. The paper does not contrast this approach with Tree of Thought approaches, or other methods that trade accuracy for inference timesteps. The shown baselines are quite weak and stale at this point in the field. 2. The problems being attacked by this paper would benefit significantly from tool use, which is not a mainstay of the paper. This lack of treatment reduces the impact and novelty of this method. 3. The paper's writing style can be improved to get to the core of the training methodolo
- Integrating both inductive and deductive reasoning within LLM is a very interesting challenge.
- The introduction is quite long and I think it could be reduced to leave more space to more meaningful details about the method and the experiments. - The description of the assumptions is neither informally nor formally clear, and some symbols have not been correctly defined (see questions). OTHER COMMENTS: - "(..) novel approach designed to enhance LLM reasoning by integrating both inductive and deductive reasoning processes within the prompt construction framework as the Figure 1 shows."
1. Adaptive reasoning is an important challenge for LLMs. The proposed approach is novel. 2. This paper selects three tasks ranging from various complexity, showing its improvement in flexibility and adaptability of LLMs in complex problem-solving tasks.
1. Introduction is too long and does not say much. 2. Poor presentation - Figure 1 is ambiguous. how complexity is measured? And why models are arranged in x-axis (complexity)? - Figure 2 does not help understanding since the concept of "Evolved Hypotheses" in DID part is still too general. Need more information to demonstrate the overall framework. Illustrations of "Knowledge Base", "Extracted Patterns and Facts" seem unnecessary here. 3. Need to improve the orgranization of methodology. Cur
The paper presents the Deductive and InDuctive (DID) method, which offers an approach to enhancing the reasoning capabilities of Large Language Models (LLMs) by integrating deductive and inductive reasoning within the prompt construction process. This method is inspired by cognitive science, aligning with how humans adapt their reasoning strategies.
- The insufficiency of the experimental validation is one of the paper's main limitations. The paper only provides empirical validation on three relatively simple datasets and lacks in-depth analysis. On one hand, the authors should consider validating their method on more realistic datasets to demonstrate the effectiveness and generalizability of the proposed approach. On the other hand, benchmarking the DID method against state-of-the-art methods on the same tasks would help position the DID m
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques