Aristotle: Mastering Logical Reasoning with A Logic-Complete Decompose-Search-Resolve Framework

TL;DR

Aristotle introduces a comprehensive reasoning framework that leverages logical structure throughout the process, significantly improving accuracy and efficiency in logical reasoning tasks for large language models.

Contribution

It presents a novel, logic-complete framework with decomposition, search, and resolution components, enhancing logical reasoning in LLMs beyond existing methods.

Findings

Outperforms state-of-the-art reasoning frameworks in accuracy

Achieves higher efficiency in complex logical reasoning tasks

Reduces sub-task complexity and search errors

Abstract

In the context of large language models (LLMs), current advanced reasoning methods have made impressive strides in various reasoning tasks. However, when it comes to logical reasoning tasks, major challenges remain in both efficacy and efficiency. This is rooted in the fact that these systems fail to fully leverage the inherent structure of logical tasks throughout the reasoning processes such as decomposition, search, and resolution. To address this, we propose a logic-complete reasoning framework, Aristotle, with three key components: Logical Decomposer, Logical Search Router, and Logical Resolver. In our framework, symbolic expressions and logical rules are comprehensively integrated into the entire reasoning process, significantly alleviating the bottlenecks of logical reasoning, i.e., reducing sub-task complexity, minimizing search errors, and resolving logical contradictions. The experimental results on several datasets demonstrate that Aristotle consistently outperforms state-of-the-art reasoning frameworks in both accuracy and efficiency, particularly excelling in complex logical reasoning scenarios. We will open-source all our code at https://llm-symbol.github.io/Aristotle/.