Variational Inference MPC using Tsallis Divergence

TL;DR

This paper introduces a novel Variational Inference-Model Predictive Control framework based on Tsallis divergence, enabling improved control performance and risk sensitivity analysis across multiple robotic systems.

Contribution

It develops a generalized Tsallis divergence-based variational inference algorithm that unifies and extends existing control methods with superior cost variance reduction.

Findings

Enhanced control performance in robotic systems.

Effective cost and reward transformation capabilities.

Theoretical and numerical validation of risk sensitivity.

Abstract

In this paper, we provide a generalized framework for Variational Inference-Stochastic Optimal Control by using thenon-extensive Tsallis divergence. By incorporating the deformed exponential function into the optimality likelihood function, a novel Tsallis Variational Inference-Model Predictive Control algorithm is derived, which includes prior works such as Variational Inference-Model Predictive Control, Model Predictive PathIntegral Control, Cross Entropy Method, and Stein VariationalInference Model Predictive Control as special cases. The proposed algorithm allows for effective control of the cost/reward transform and is characterized by superior performance in terms of mean and variance reduction of the associated cost. The aforementioned features are supported by a theoretical and numerical analysis on the level of risk sensitivity of the proposed algorithm as well as simulation experiments on 5 different robotic systems with 3 different policy parameterizations.