Model Predictive Path Integral Control as Preconditioned Gradient Descent

TL;DR

This paper provides a new optimization-theoretic interpretation of Model Predictive Path Integral (MPPI) control, revealing its structure as a preconditioned gradient descent method, and offers convergence guarantees supported by numerical experiments.

Contribution

It introduces a variational interpretation of MPPI as a KL-regularized problem, connecting it to preconditioned gradient descent and enabling convergence analysis.

Findings

Classical MPPI is a special case of preconditioned gradient descent.

Explicit convergence bounds are derived for MPPI under certain conditions.

Numerical experiments validate the theoretical insights and hyperparameter effects.

Abstract

Model Predictive Path Integral (MPPI) control is a popular sampling-based method for trajectory optimization in nonlinear and nonconvex settings, yet its optimization structure remains only partially understood. We develop a variational, optimization-theoretic interpretation of MPPI by lifting constrained trajectory optimization to a KL-regularized problem over distributions and reducing it to a negative log-partition (free-energy) objective over a tractable sampling family. For a general parametric family, this yields a preconditioned gradient method on the distribution parameters and a natural multi-step extension of MPPI. For the fixed-covariance Gaussian family, we show that classical MPPI is recovered exactly as a preconditioned gradient descent step with unit step size. This interpretation enables a direct convergence analysis: under bounded feasible sets, we derive an explicit upper bound on the smoothness constant and a simple sufficient condition guaranteeing descent of exact MPPI. Numerical experiments support the theory and illustrate the effect of key hyperparameters on performance.