Optimizing Prompt Sequences using Monte Carlo Tree Search for LLM-Based Optimization

TL;DR

This paper introduces MCTS-OPS, a neural-symbolic framework that uses Monte Carlo Tree Search to optimize prompt sequences, significantly improving code generation and problem-solving in complex tasks with LLMs.

Contribution

It presents a novel MCTS-guided prompt optimization method that enhances LLM performance on complex, multi-step tasks, especially in code generation and optimization.

Findings

2-4x higher reward in network optimization tasks

3x lower standard deviation in results

10% increase in chance of finding the optimal solution

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities in code generation and structured reasoning; however, their performance often degrades on complex tasks that require consistent multi-step planning. Recent work has explored combining LLMs with Monte Carlo Tree Search (MCTS), yet existing approaches primarily focus on generating heuristic-based code for optimization or target simpler tasks where correctness alone is sufficient. In this work, we propose MCTS-OPS, a novel neural-symbolic framework that formulates prompt selection as a sequential decision process guided by MCTS. Our method explores and refines multi-step prompt sequences for the goal of improving code generation quality and enhancing the problem-solving capabilities of LLMs in general optimization. Experiments on network optimization show significant improvement over the baselines, both in the success rate of executing the generated code and in the optimization results with the specified objective and constraints (2$\sim$4$\times$ higher reward and 3$\times$ lower standard deviation). Moreover, it improves the chance of attaining the optimal solution by about 10\% of cases, compared to baseline methods in hard problems. These results highlight the promise of combining symbolic planning with LLMs for robust, high-quality code generation in complex domains.