PODDP: Partially Observable Differential Dynamic Programming for Latent Belief Space Planning

TL;DR

This paper introduces PODDP, an efficient planning algorithm for partially observable systems with discrete and continuous uncertainties, enabling robust trajectory planning in complex robotic scenarios.

Contribution

It develops a novel differential dynamic programming approach for belief space planning with latent discrete states and nonlinear dynamics, including hierarchical extensions.

Findings

Outperforms heuristic planning methods in benchmarks.

Enables robust autonomous lane changing with uncertain interactions.

Handles uncertainty over dynamics, goals, and other agents' intentions.

Abstract

Autonomous agents are limited in their ability to observe the world state. Partially observable Markov decision processes (POMDPs) formally model the problem of planning under world state uncertainty, but POMDPs with continuous actions and nonlinear dynamics suitable for robotics applications are challenging to solve. In this paper, we present an efficient differential dynamic programming (DDP) algorithm for belief space planning in POMDPs with uncertainty over a discrete latent state, and continuous states, actions, observations, and nonlinear dynamics. This representation allows planning of dynamic trajectories which are sensitive to structured uncertainty over discrete latent world states. We develop dynamic programming techniques to optimize a contingency plan over a tree of possible observations and belief space trajectories, and also derive a hierarchical version of the algorithm. Our method is applicable to problems with uncertainty over the cost or reward function (e.g., the configuration of goals or obstacles), uncertainty over the dynamics (e.g., the dynamical mode of a hybrid system), and uncertainty about interactions, where other agents' behavior is conditioned on latent intentions. Benchmarks show that our algorithm outperforms popular heuristic approaches to planning under uncertainty, and results from an autonomous lane changing task demonstrate that our algorithm can synthesize robust interactive trajectories.