Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation

TL;DR

This paper introduces 'Everything of Thoughts' (XoT), a novel prompting method for LLMs that integrates reinforcement learning and MCTS to improve performance, efficiency, and flexibility in complex problem-solving.

Contribution

XoT defies existing thought paradigms by combining external domain knowledge with LLMs, enabling autonomous, high-quality, and flexible problem-solving with minimal interactions.

Findings

XoT outperforms existing methods on multi-solution tasks.

XoT generates multiple solutions with only one LLM call.

XoT demonstrates strong generalization to unseen problems.

Abstract

Recent advancements in Large Language Models (LLMs) have revolutionized decision-making by breaking down complex problems into more manageable language sequences referred to as "thoughts". An effective thought design should consider three key perspectives: performance, efficiency, and flexibility. However, existing thought can at most exhibit two of these attributes. To address these limitations, we introduce a novel thought prompting approach called "Everything of Thoughts" (XoT) to defy the law of "Penrose triangle of existing thought paradigms. XoT leverages pretrained reinforcement learning and Monte Carlo Tree Search (MCTS) to incorporate external domain knowledge into thoughts, thereby enhancing LLMs' capabilities and enabling them to generalize to unseen problems efficiently. Through the utilization of the MCTS-LLM collaborative thought revision framework, this approach autonomously produces high-quality comprehensive cognitive mappings with minimal LLM interactions. Additionally, XoT empowers LLMs to engage in unconstrained thinking, allowing for flexible cognitive mappings for problems with multiple solutions. We evaluate XoT on several challenging multi-solution problem-solving tasks, including Game of 24, 8-Puzzle, and Pocket Cube. Our results demonstrate that XoT significantly outperforms existing approaches. Notably, XoT can yield multiple solutions with just one LLM call, showcasing its remarkable proficiency in addressing complex problems across diverse domains.