# Adaptive Thompson Sampling Stacks for Memory Bounded Open-Loop Planning

## TL;DR

This paper introduces SYMBOL, an adaptive memory-bounded open-loop planning method using a stack of Thompson Sampling bandits, which is effective across large POMDP benchmarks and adapts memory use without prior domain knowledge.

## Contribution

The paper presents SYMBOL, a novel adaptive planning approach that maintains a memory-bounded stack of Thompson Sampling bandits, improving robustness and efficiency in POMDP planning.

## Key findings

- Effective in large POMDP benchmarks
- Robust to hyperparameter choices
- Adaptive memory consumption demonstrated

## Abstract

We propose Stable Yet Memory Bounded Open-Loop (SYMBOL) planning, a general memory bounded approach to partially observable open-loop planning. SYMBOL maintains an adaptive stack of Thompson Sampling bandits, whose size is bounded by the planning horizon and can be automatically adapted according to the underlying domain without any prior domain knowledge beyond a generative model. We empirically test SYMBOL in four large POMDP benchmark problems to demonstrate its effectiveness and robustness w.r.t. the choice of hyperparameters and evaluate its adaptive memory consumption. We also compare its performance with other open-loop planning algorithms and POMCP.