SR-FoT: A Syllogistic-Reasoning Framework of Thought for Large Language Models Tackling Knowledge-based Reasoning Tasks

TL;DR

SR-FoT is a multi-stage framework that enhances large language models' deductive reasoning by mimicking human syllogistic reasoning, improving their ability to handle complex knowledge-based tasks.

Contribution

It introduces a novel syllogistic reasoning framework that guides LLMs through multi-stage deductive reasoning, addressing limitations of chain-of-thought prompts.

Findings

Significantly improves reasoning accuracy on knowledge-based tasks.

Outperforms baseline methods in deductive reasoning benchmarks.

Demonstrates robustness across various complex reasoning scenarios.

Abstract

Deductive reasoning is a crucial logical capability that assists us in solving complex problems based on existing knowledge. Although augmented by Chain-of-Thought prompts, Large Language Models (LLMs) might not follow the correct reasoning paths. Enhancing the deductive reasoning abilities of LLMs, and leveraging their extensive built-in knowledge for various reasoning tasks, remains an open question. Attempting to mimic the human deductive reasoning paradigm, we propose a multi-stage Syllogistic-Reasoning Framework of Thought (SR-FoT) that enables LLMs to perform syllogistic deductive reasoning to handle complex knowledge-based reasoning tasks. Our SR-FoT begins by interpreting the question and then uses the interpretation and the original question to propose a suitable major premise. It proceeds by generating and answering minor premise questions in two stages to match the minor premises. Finally, it guides LLMs to use the previously generated major and minor premises to perform syllogistic deductive reasoning to derive the answer to the original question. Extensive and thorough experiments on knowledge-based reasoning tasks have demonstrated the effectiveness and advantages of our SR-FoT.