Optimality and Suboptimality of MPPI Control in Stochastic and Deterministic Settings

TL;DR

This paper analyzes the optimality and suboptimality of MPPI control in stochastic and deterministic systems, providing theoretical insights and numerical illustrations to guide its application in control tasks.

Contribution

It introduces a framework connecting MPPI to optimal control problems and quantifies its suboptimality in deterministic settings, highlighting how hyperparameters influence performance.

Findings

Suboptimality grows quadratically with uncertainty in smooth, unconstrained systems.

Proper hyperparameter tuning can reduce MPPI suboptimality.

Numerical examples validate theoretical predictions.

Abstract

Model predictive path integral (MPPI) control has recently received a lot of attention, especially in the robotics and reinforcement learning communities. This letter aims to make the MPPI control framework more accessible to the optimal control community. We present three classes of optimal control problems and their solutions by MPPI. Further, we investigate the suboptimality of MPPI to general deterministic nonlinear discrete-time systems. Here, suboptimality is defined as the deviation between the control provided by MPPI and the optimal solution to the deterministic optimal control problem. Our findings are that in a smooth and unconstrained setting, the growth of suboptimality in the control input trajectory is second-order with the scaling of uncertainty. The results indicate that the suboptimality of the MPPI solution can be modulated by appropriately tuning the hyperparameters. We illustrate our findings using numerical examples.