# Robust and Adaptive Planning under Model Uncertainty

## TL;DR

This paper introduces RAMCP, a novel algorithm for risk-sensitive planning under model uncertainty, balancing exploration, exploitation, and robustness through a game-theoretic approach, with proven and empirical advantages.

## Contribution

The paper presents RAMCP, a new risk-sensitive planning algorithm that handles model uncertainty efficiently and with theoretical guarantees, advancing decision-making under uncertainty.

## Key findings

- RAMCP-F converges to an optimal risk-sensitive policy.
- RAMCP-I offers computational efficiency with comparable empirical results.
- Demonstrated effectiveness on bandit and patient treatment scenarios.

## Abstract

Planning under model uncertainty is a fundamental problem across many applications of decision making and learning. In this paper, we propose the Robust Adaptive Monte Carlo Planning (RAMCP) algorithm, which allows computation of risk-sensitive Bayes-adaptive policies that optimally trade off exploration, exploitation, and robustness. RAMCP formulates the risk-sensitive planning problem as a two-player zero-sum game, in which an adversary perturbs the agent's belief over the models. We introduce two versions of the RAMCP algorithm. The first, RAMCP-F, converges to an optimal risk-sensitive policy without having to rebuild the search tree as the underlying belief over models is perturbed. The second version, RAMCP-I, improves computational efficiency at the cost of losing theoretical guarantees, but is shown to yield empirical results comparable to RAMCP-F. RAMCP is demonstrated on an n-pull multi-armed bandit problem, as well as a patient treatment scenario.