TypedThinker: Diversify Large Language Model Reasoning with Typed Thinking

TL;DR

TypedThinker enhances large language models by predicting and applying diverse reasoning types, leading to significant performance improvements on logical and mathematical reasoning benchmarks without needing larger models.

Contribution

The paper introduces TypedThinker, a method that predicts suitable reasoning types for problems and guides LLMs to diversify their reasoning strategies, improving problem-solving performance.

Findings

Performance gains of 3.4% on Mistral 7B

Performance gains of 6.5% on LLaMA3 8B

Performance gains of 7% on Qwen 2 7B

Abstract

Large Language Models (LLMs) have demonstrated strong reasoning capabilities in solving complex problems. However, current approaches primarily enhance reasoning through the elaboration of thoughts while neglecting the diversity of reasoning types. LLMs typically employ deductive reasoning, proceeding step-by-step from given conditions, which limits their exploration during problem-solving. Our analysis reveals that certain problems are exclusively solvable through specific reasoning strategies like inductive, abductive, or analogical reasoning. However, incorporating diverse reasoning approaches presents two key challenges: identifying the appropriate reasoning type for each problem and exploiting this approach during problem-solving. Therefore, we propose the TypedThinker that predicts suitable reasoning types based on the problem and their previous effectiveness and provides relevant demonstrations to guide LLMs in applying these strategies. Experimental results show significant improvements across multiple benchmarks, with performance gains of 3.4% for Mistral 7B, 6.5% for LLaMA3 8B, and 7% for Qwen 2 7B on logical and mathematical reasoning tasks. TypedThinker enhances LLM reasoning without requiring knowledge distillation from larger models. It can be integrated into more advanced systems like GPT-4o or specialized models like MetaMath to diversify their reasoning approaches and improve their problem-solving capabilities.