Efficient Sampling in POMDPs with Lipschitz Bandits for Motion Planning in Continuous Spaces

TL;DR

This paper introduces a Lipschitz bandit approach to improve sampling efficiency in POMDP-based motion planning, leveraging outcome similarities in continuous spaces to enhance decision-making under uncertainty.

Contribution

It proposes a novel Lipschitz bandit heuristic for POMDP sampling, specifically tailored for motion planning in continuous spaces, improving efficiency over traditional methods.

Findings

Enhanced sampling efficiency demonstrated in automated driving scenarios.

Lipschitz assumptions lead to better exploration and decision quality.

Applicable to continuous space motion planning problems.

Abstract

Decision making under uncertainty can be framed as a partially observable Markov decision process (POMDP). Finding exact solutions of POMDPs is generally computationally intractable, but the solution can be approximated by sampling-based approaches. These sampling-based POMDP solvers rely on multi-armed bandit (MAB) heuristics, which assume the outcomes of different actions to be uncorrelated. In some applications, like motion planning in continuous spaces, similar actions yield similar outcomes. In this paper, we utilize variants of MAB heuristics that make Lipschitz continuity assumptions on the outcomes of actions to improve the efficiency of sampling-based planning approaches. We demonstrate the effectiveness of this approach in the context of motion planning for automated driving.