Hybrid control trajectory optimization under uncertainty

TL;DR

This paper introduces a DDP-based method for hybrid control trajectory optimization that efficiently handles discrete-continuous control combinations and extends to uncertain, partially observable environments.

Contribution

It presents a novel approach integrating discrete actions into DDP and extends it to POMDPs for planning under uncertainty.

Findings

Successfully optimized gear switching in a car driving task.

Effectively planned box pushing with unknown pose and friction.

Demonstrated robustness in uncertain, partially observable scenarios.

Abstract

Trajectory optimization is a fundamental problem in robotics. While optimization of continuous control trajectories is well developed, many applications require both discrete and continuous, i.e., hybrid, controls. Finding an optimal sequence of hybrid controls is challenging due to the exponential explosion of discrete control combinations. Our method, based on Differential Dynamic Programming (DDP), circumvents this problem by incorporating discrete actions inside DDP: we first optimize continuous mixtures of discrete actions, and, subsequently force the mixtures into fully discrete actions. Moreover, we show how our approach can be extended to partially observable Markov decision processes (POMDPs) for trajectory planning under uncertainty. We validate the approach in a car driving problem where the robot has to switch discrete gears and in a box pushing application where the robot can switch the side of the box to push. The pose and the friction parameters of the pushed box are initially unknown and only indirectly observable.