Real-time Sampling-based Model Predictive Control based on Reverse Kullback-Leibler Divergence and Its Adaptive Acceleration

TL;DR

This paper introduces a novel sampling-based MPC approach using reverse Kullback-Leibler divergence and adaptive Nesterov acceleration, enabling real-time control of complex robotic systems with improved convergence and task versatility.

Contribution

It proposes a new reverse KL divergence-based sampling method with an adaptive acceleration scheme, enhancing real-time robotic control performance and convergence over traditional methods.

Findings

Wider variety of tasks solved statistically

Higher degrees-of-freedom tasks handled with CPU only

Improved real-time control in variable impedance robot

Abstract

Sampling-based model predictive control (MPC) has the potential for use in a wide variety of robotic systems. However, its unstable updates and poor convergence render it unsuitable for real-time control of robotic systems. This study addresses this challenge with a novel approach from reverse Kullback-Leibler divergence, which has a mode-seeking property and is likely to find one of the locally optimal solutions early. Using this approach, a weighted maximum likelihood estimation with positive and negative weights is obtained and solved using the mirror descent (MD) algorithm. Negative weights eliminate unnecessary actions, but a practical implementation needs to be designed to avoid interference with positive and negative updates based on rejection sampling. In addition, Nesterov's acceleration method for the proposed MD is modified to improve heuristic step size adaptive to the noise estimated in update amounts. Real-time simulations show that the proposed method can solve a wider variety of tasks statistically than the conventional method. In addition, higher degrees-of-freedom tasks can be solved by the improved acceleration even with a CPU only. The real-world applicability of the proposed method is also demonstrated by optimizing the operability in a variable impedance control of a force-driven mobile robot. https://youtu.be/D8bFMzct1XM