Continuous Monte Carlo Graph Search
Kalle Kujanp\"a\"a, Amin Babadi, Yi Zhao, Juho Kannala, Alexander, Ilin, Joni Pajarinen

TL;DR
This paper introduces Continuous Monte Carlo Graph Search (CMCGS), a novel online planning method for continuous environments that improves efficiency and scalability over existing MCTS extensions by sharing policies across similar states.
Contribution
The paper proposes CMCGS, which clusters states into stochastic action nodes to create a graph structure, enabling efficient planning in continuous spaces and outperforming existing methods.
Findings
Outperforms existing planning methods on DeepMind Control Suite benchmarks.
Scales effectively through parallelization.
Outperforms CEM in continuous control with learned dynamics.
Abstract
Online planning is crucial for high performance in many complex sequential decision-making tasks. Monte Carlo Tree Search (MCTS) employs a principled mechanism for trading off exploration for exploitation for efficient online planning, and it outperforms comparison methods in many discrete decision-making domains such as Go, Chess, and Shogi. Subsequently, extensions of MCTS to continuous domains have been developed. However, the inherent high branching factor and the resulting explosion of the search tree size are limiting the existing methods. To address this problem, we propose Continuous Monte Carlo Graph Search (CMCGS), an extension of MCTS to online planning in environments with continuous state and action spaces. CMCGS takes advantage of the insight that, during planning, sharing the same action policy between several states can yield high performance. To implement this idea, at…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGraph Theory and Algorithms · Semantic Web and Ontologies · Advanced Database Systems and Queries
