Self-guided Knowledgeable Network of Thoughts: Amplifying Reasoning with Large Language Models

TL;DR

The paper introduces Knowledgeable Network of Thoughts (kNoT), a novel prompt scheme that enhances large language model reasoning by enabling executable, network-structured plans, outperforming existing methods with less prompt engineering.

Contribution

It proposes the LLM Workflow Template (LWT), allowing for flexible, reliable, network-based reasoning plans that improve performance and reduce prompt engineering compared to prior paradigms.

Findings

kNoT achieves 92% accuracy in sorting 32 numbers.

Reduces task-specific prompts by up to 87%.

Outperforms state-of-the-art methods on six use cases.

Abstract

We introduce Knowledgeable Network of Thoughts (kNoT): a prompt scheme that advances the capabilities of large language models (LLMs) beyond existing paradigms like Chain-of-Thought (CoT), Tree of Thoughts (ToT), and Graph of Thoughts (GoT). The key innovation of kNoT is the LLM Workflow Template (LWT), which allows for an executable plan to be specified by LLMs for LLMs. LWT allows these plans to be arbitrary networks, where single-step LLM operations are nodes, and edges correspond to message passing between these steps. Furthermore, LWT supports selection of individual elements through indexing, facilitating kNoT to produce intricate plans where each LLM operation can be limited to elementary operations, greatly enhancing reliability over extended task sequences. We demonstrate that kNoT significantly outperforms the state of the art on six use cases, while reducing the need for extensive prompt engineering. For instance, kNoT finds 92% accuracy for sorting 32 numbers over 12% and 31% for ToT and GoT, while utilizing up to 84.4% and 87.3% less task-specific prompts, respectively.