CoVO-MPC: Theoretical Analysis of Sampling-based MPC and Optimal Covariance Design

TL;DR

This paper provides a theoretical analysis of sampling-based MPC, specifically MPPI, establishing convergence properties and introducing CoVo-MPC, which optimally adjusts sampling covariance to enhance convergence and performance.

Contribution

The paper offers the first convergence analysis of MPPI and introduces CoVo-MPC, a novel algorithm that optimally schedules sampling covariance for improved convergence and control.

Findings

MPPI has at least linear convergence for quadratic optimization.

CoVo-MPC outperforms standard MPPI by 43-54% in simulations.

CoVo-MPC demonstrates superior real-world quadrotor control performance.

Abstract

Sampling-based Model Predictive Control (MPC) has been a practical and effective approach in many domains, notably model-based reinforcement learning, thanks to its flexibility and parallelizability. Despite its appealing empirical performance, the theoretical understanding, particularly in terms of convergence analysis and hyperparameter tuning, remains absent. In this paper, we characterize the convergence property of a widely used sampling-based MPC method, Model Predictive Path Integral Control (MPPI). We show that MPPI enjoys at least linear convergence rates when the optimization is quadratic, which covers time-varying LQR systems. We then extend to more general nonlinear systems. Our theoretical analysis directly leads to a novel sampling-based MPC algorithm, CoVariance-Optimal MPC (CoVo-MPC) that optimally schedules the sampling covariance to optimize the convergence rate. Empirically, CoVo-MPC significantly outperforms standard MPPI by 43-54% in both simulations and real-world quadrotor agile control tasks. Videos and Appendices are available at \url{https://lecar-lab.github.io/CoVO-MPC/}.