Q-Guided Stein Variational Model Predictive Control via RL-informed Policy Prior

TL;DR

This paper introduces Q-SVMPC, a novel learning-based MPC method that uses Stein variational inference guided by learned Q-values to maintain solution diversity and improve robustness in complex control tasks.

Contribution

It presents a new Q-guided Stein variational MPC approach with an RL-informed policy prior, enhancing solution diversity and stability over existing methods.

Findings

Improved sample efficiency in navigation and manipulation tasks

Enhanced stability and robustness compared to baseline methods

Successful real-world application in fruit-picking task

Abstract

Model Predictive Control (MPC) enables reliable trajectory optimization under dynamics constraints, but often depends on accurate dynamics models and carefully hand-designed cost functions. Recent learning-based MPC methods aim to reduce these modeling and cost-design burdens by learning dynamics, priors, or value-related guidance signals. Yet many existing approaches still rely on deterministic gradient-based solvers (e.g., differentiable MPC) or parametric sampling-based updates (e.g., CEM/MPPI), which can lead to mode collapse and convergence to a single dominant solution. We propose Q-SVMPC, a Q-guided Stein variational MPC method with an RL-informed policy prior, which casts learning-based MPC as trajectory-level posterior inference and refines trajectory particles via SVGD under learned soft Q-value guidance to explicitly preserve diverse solutions. Experiments on navigation, robotic manipulation, and a real-world fruit-picking task show improved sample efficiency, stability, and robustness over MPC, model-free RL, and learning-based MPC baselines.