Importance sampling-based approximate optimal planning and control

TL;DR

This paper introduces an importance sampling-based method for approximate optimal control of nonlinear systems, enabling scalable, robust motion planning by iteratively refining policy weights through adaptive search.

Contribution

It develops a novel importance sampling approach for iterative policy weight optimization in nonlinear control, incorporating trajectory verification for robustness.

Findings

Method effectively computes near-optimal policies for nonlinear systems.

Numerical experiments demonstrate robustness and efficiency.

Applicable to systems like Dubins car with nonlinear costs.

Abstract

In this paper, we propose a sampling-based planning and optimal control method of nonlinear systems under non-differentiable constraints. Motivated by developing scalable planning algorithms, we consider the optimal motion plan to be a feedback controller that can be approximated by a weighted sum of given bases. Given this approximate optimal control formulation, our main contribution is to introduce importance sampling, specifically, model-reference adaptive search algorithm, to iteratively compute the optimal weight parameters, i.e., the weights corresponding to the optimal policy function approximation given chosen bases. The key idea is to perform the search by iteratively estimating a parametrized distribution which converges to a Dirac's Delta that infinitely peaks on the global optimal weights. Then, using this direct policy search, we incorporated trajectory-based verification to ensure that, for a class of nonlinear systems, the obtained policy is not only optimal but robust to bounded disturbances. The correctness and efficiency of the methods are demonstrated through numerical experiments including linear systems with a nonlinear cost function and motion planning for a Dubins car.