Pheromone-based Learning of Optimal Reasoning Paths

TL;DR

This paper introduces ACO-ToT, a biologically inspired algorithm combining ant colony optimization with large language models to efficiently discover optimal reasoning paths for complex problems, outperforming existing methods.

Contribution

It presents a novel ACO-guided Tree of Thought algorithm that enhances reasoning in LLMs by using pheromone trails and collective search inspired by neurological systems.

Findings

ACO-ToT outperforms existing chain-of-thought methods on challenging reasoning tasks.

The method effectively reinforces productive reasoning paths through pheromone updates.

Incorporating collective search mechanisms significantly improves LLM reasoning capabilities.

Abstract

Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities through chain-of-thought prompting, yet discovering effective reasoning methods for complex problems remains challenging due to the vast space of possible intermediate steps. We introduce Ant Colony Optimization-guided Tree of Thought (ACO-ToT), a novel algorithm that combines ACO with LLMs to discover optimal reasoning paths for complex problems efficiently. Drawing inspiration from Hebbian learning in neurological systems, our method employs a collection of distinctly fine-tuned LLM "ants" to traverse and lay pheromone trails through a centralized tree of thought, with each ant's movement governed by a weighted combination of existing pheromone trails and its own specialized expertise. The algorithm evaluates complete reasoning paths using a mixture-of-experts-based scoring function, with pheromones reinforcing productive reasoning paths across iterations. Experiments on three challenging reasoning tasks (GSM8K, ARC-Challenge, and MATH) demonstrate that ACO-ToT performs significantly better than existing chain-of-thought optimization approaches, suggesting that incorporating biologically inspired collective search mechanisms into LLM inference can substantially enhance reasoning capabilities.