Optimized Monte Carlo Tree Search for Enhanced Decision Making in the FrozenLake Environment

TL;DR

This paper introduces an optimized Monte Carlo Tree Search algorithm tailored for the FrozenLake environment, improving decision-making efficiency and success rates in stochastic reinforcement learning tasks.

Contribution

The paper presents a novel optimized MCTS implementation that enhances performance in stochastic environments by integrating cumulative reward, visit counts, and UCT, outperforming baseline algorithms.

Findings

Optimized MCTS achieves higher rewards in FrozenLake.

The approach converges faster than traditional methods.

Outperforms Q-Learning and policy-based algorithms in stochastic settings.

Abstract

Monte Carlo Tree Search (MCTS) is a powerful algorithm for solving complex decision-making problems. This paper presents an optimized MCTS implementation applied to the FrozenLake environment, a classic reinforcement learning task characterized by stochastic transitions. The optimization leverages cumulative reward and visit count tables along with the Upper Confidence Bound for Trees (UCT) formula, resulting in efficient learning in a slippery grid world. We benchmark our implementation against other decision-making algorithms, including MCTS with Policy and Q-Learning, and perform a detailed comparison of their performance. The results demonstrate that our optimized approach effectively maximizes rewards and success rates while minimizing convergence time, outperforming baseline methods, especially in environments with inherent randomness.