Advancing Trajectory Optimization with Approximate Inference: Exploration, Covariance Control and Adaptive Risk

TL;DR

This paper enhances stochastic trajectory optimization by framing control as inference, introducing an input inference algorithm with an expert controller, adaptive risk sensitivity, and covariance control capabilities for nonlinear systems.

Contribution

It introduces an advanced input inference algorithm that combines open-loop and closed-loop benefits, adaptive risk sensitivity, and covariance control with minimal modifications.

Findings

The expert controller improves trajectory optimization in nonlinear systems.

The inference approach provides inherent adaptive risk sensitivity.

The method enables covariance control with minor algorithmic changes.

Abstract

Discrete-time stochastic optimal control remains a challenging problem for general, nonlinear systems under significant uncertainty, with practical solvers typically relying on the certainty equivalence assumption, replanning and/or extensive regularization. Control as inference is an approach that frames stochastic control as an equivalent inference problem, and has demonstrated desirable qualities over existing methods, namely in exploration and regularization. We look specifically at the input inference for control (i2c) algorithm, and derive three key characteristics that enable advanced trajectory optimization: An `expert' linear Gaussian controller that combines the benefits of open-loop optima and closed-loop variance reduction when optimizing for nonlinear systems, inherent adaptive risk sensitivity from the inference formulation, and covariance control functionality with only a minor algorithmic adjustment.