CoAT: Chain-of-Associated-Thoughts Framework for Enhancing Large Language Models Reasoning

TL;DR

The paper introduces CoAT, a novel framework combining Monte Carlo Tree Search and associative memory to improve reasoning in large language models by exploring diverse pathways and dynamically updating knowledge.

Contribution

It presents a new Chain-of-Associated-Thoughts framework that enhances LLM reasoning by integrating structured exploration with adaptive knowledge updating.

Findings

Over 10% performance improvement on multi-hop reasoning datasets

More than 15% gain on proprietary CRB dataset

Effective exploration of diverse reasoning pathways

Abstract

Research on LLM technologies is rapidly emerging, with most of them employ a 'fast thinking' approach to inference. Most LLMs generate the final result based solely on a single query and LLM's reasoning capabilities. However, with the advent of OpenAI-o1, 'slow thinking' techniques have garnered increasing attention because its process is closer to the human thought process. Inspired by the human ability to constantly associate and replenish knowledge during thinking, we developed the novel Chain-of-Associated-Thoughts (CoAT) framework, which introduces an innovative synergy between the Monte Carlo Tree Search (MCTS) algorithm and a dynamic mechanism for integrating new key information, termed 'associative memory'. By combining the structured exploration capabilities of MCTS with the adaptive learning capacity of associative memory, CoAT significantly expands the LLM search space, enabling our framework to explore diverse reasoning pathways and dynamically update its knowledge base in real-time. This allows the framework to not only revisit and refine earlier inferences but also adaptively incorporate evolving information, ensuring that the final output is both accurate and comprehensive. We validate CoAT's effectiveness across a variety of generative and reasoning tasks. Quantitative experiments show that CoAT achieves over 10% performance improvement on open-source multi-hop reasoning datasets (HotpotQA, MuSiQue) and more than 15% gain on our proprietary CRB dataset.