Sampling-Based Control via Entropy-Regularized Optimal Transport

TL;DR

OT-MPC introduces an entropy-regularized optimal transport approach to sampling-based control, improving real-time nonlinear robotic system performance by refining control samples and maintaining solution diversity.

Contribution

It proposes OT-MPC, a novel sampling-based control algorithm using optimal transport and Sinkhorn updates for better handling complex cost landscapes.

Findings

OT-MPC outperforms existing methods in navigation, manipulation, and locomotion tasks.

It maintains solution diversity while refining control candidates.

Real-time performance is achieved through closed-form Sinkhorn updates.

Abstract

Sampling-based model predictive control methods like MPPI and CEM are essential for real-time control of nonlinear robotic systems, particularly where discontinuous dynamics preclude gradient-based optimization. However, these methods derive from information-theoretic objectives that are agnostic to the geometry of the control problem, leading to pathological behaviors such as mode-averaging when the cost landscape is complex. We present OT-MPC, a sampling-based algorithm that overcomes these limitations through an entropy-regularized optimal transport formulation. By computing an optimal coupling between candidate control sequences and low-cost proposals, OT-MPC refines candidates toward nearby promising samples while coordinating updates across the ensemble to maintain coverage of the solution space. We derive closed-form, gradient-free updates via the Sinkhorn algorithm, enabling real-time performance. Experiments on navigation, manipulation, and locomotion tasks demonstrate improved success rates over existing methods.