Plan-MCTS: Plan Exploration for Action Exploitation in Web Navigation

TL;DR

Plan-MCTS introduces a semantic plan space and dual gating to improve web navigation efficiency and accuracy for autonomous agents using LLMs, addressing exploration and noise challenges.

Contribution

It proposes a novel framework that reformulates web navigation into a dense plan tree with semantic abstraction, enhancing exploration and robustness over prior methods.

Findings

Achieves state-of-the-art performance on WebArena.

Outperforms existing approaches in task success rate.

Demonstrates higher search efficiency and robustness.

Abstract

Large Language Models (LLMs) have empowered autonomous agents to handle complex web navigation tasks. While recent studies integrate tree search to enhance long-horizon reasoning, applying these algorithms in web navigation faces two critical challenges: sparse valid paths that lead to inefficient exploration, and a noisy context that dilutes accurate state perception. To address this, we introduce Plan-MCTS, a framework that reformulates web navigation by shifting exploration to a semantic Plan Space. By decoupling strategic planning from execution grounding, it transforms sparse action space into a Dense Plan Tree for efficient exploration, and distills noisy contexts into an Abstracted Semantic History for precise state awareness. To ensure efficiency and robustness, Plan-MCTS incorporates a Dual-Gating Reward to strictly validate both physical executability and strategic alignment and Structural Refinement for on-policy repair of failed subplans. Extensive experiments on WebArena demonstrate that Plan-MCTS achieves state-of-the-art performance, surpassing current approaches with higher task effectiveness and search efficiency.