Reinforced Graph of Thoughts: RL-Driven Adaptive Prompting for LLMs

TL;DR

This paper introduces Reinforced Graph of Thoughts (RGoT), an RL-based method that automates the creation of adaptive, task-specific graphs of operations to improve problem solving with large language models.

Contribution

It presents a novel RL-driven approach to automatically generate adaptive graphs of operations, enhancing the flexibility of the Graph of Thoughts prompting paradigm.

Findings

Graphs of operations can be constructed adaptively based on task complexity.

The method demonstrates potential for automated, flexible problem solving with LLMs.

Results show the approach's effectiveness under certain constraints.

Abstract

Graph of Thoughts (GoT), a generalized form of recent prompting paradigms for large language models (LLMs), has been shown to be useful for elaborate problem solving. By executing a graph of operations, thoughts of the LLM are structured as an arbitrary graph, forming the actual graph of thoughts. Originally, the graph of operations is defined manually, which requires in-depth knowledge about the solution of the problem to solve. Such a static graph of operations is rigid and therefore lacks adaptability. We propose Reinforced Graph of Thoughts (RGoT), an automated approach to the GoT prompting paradigm that leverages reinforcement learning (RL) to adaptively generate a graph of operations from a human-defined set. Results indicate that, under certain constraints, it is possible to construct graphs of operations adaptively to the task's complexity in an automated way.