Monte Carlo Planning in Hybrid Belief POMDPs

TL;DR

This paper introduces HB-MCP, a Monte Carlo Tree Search-based algorithm designed to plan in hybrid belief POMDPs that involve both discrete and continuous uncertainties, addressing computational challenges in complex environments.

Contribution

The work presents a novel algorithm that extends Monte Carlo planning to hybrid belief POMDPs, incorporating UCB exploration to manage hypothesis growth effectively.

Findings

Successfully handles multi-modal beliefs in aliased environments

Demonstrates improved hypothesis management with UCB-guided growth

Effective in complex simulated scenarios with unresolved data association

Abstract

Real-world problems often require reasoning about hybrid beliefs, over both discrete and continuous random variables. Yet, such a setting has hardly been investigated in the context of planning. Moreover, existing online Partially Observable Markov Decision Processes (POMDPs) solvers do not support hybrid beliefs directly. In particular, these solvers do not address the added computational burden due to an increasing number of hypotheses with the planning horizon, which can grow exponentially. As part of this work, we present a novel algorithm, Hybrid Belief Monte Carlo Planning (HB-MCP) that utilizes the Monte Carlo Tree Search (MCTS) algorithm to solve a POMDP while maintaining a hybrid belief. We illustrate how the upper confidence bound (UCB) exploration bonus can be leveraged to guide the growth of hypotheses trees alongside the belief trees. We then evaluate our approach in highly aliased simulated environments where unresolved data association leads to multi-modal belief hypotheses.