Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up

TL;DR

This paper introduces Reversal of Thought (RoT), a cost-effective, plug-and-play framework that enhances large language models' logical reasoning during warm-up by using preference-guided reverse reasoning and meta-cognitive strategies.

Contribution

RoT is a novel reasoning framework that improves LLMs' logical reasoning by integrating preference-guided reverse reasoning and meta-cognitive mechanisms during warm-up.

Findings

RoT outperforms existing methods in reasoning accuracy.

RoT improves reasoning efficiency.

RoT effectively expands LLMs' reasoning capabilities.

Abstract

Large language models (LLMs) have shown remarkable performance in reasoning tasks but face limitations in mathematical and complex logical reasoning. Existing methods to improve LLMs' logical capabilities either involve traceable or verifiable logical sequences that generate more reliable responses by constructing logical structures yet increase computational costs, or introduces rigid logic template rules, reducing flexibility. In this paper, we propose Reversal of Thought (RoT), a plug-and-play and cost-effective reasoning framework designed to enhance the logical reasoning abilities of LLMs during the warm-up phase prior to batch inference. RoT utilizes a Preference-Guided Reverse Reasoning warm-up strategy, which integrates logical symbols for pseudocode planning through meta-cognitive mechanisms and pairwise preference self-evaluation to generate task-specific prompts solely through demonstrations, aligning with LLMs' cognitive preferences shaped by RLHF. Through reverse reasoning, we utilize a Cognitive Preference Manager to assess knowledge boundaries and further expand LLMs' reasoning capabilities by aggregating solution logic for known tasks and stylistic templates for unknown tasks. Experiments across various tasks demonstrate that RoT surpasses existing baselines in both reasoning accuracy and efficiency.