Mutual Information Optimal Density Control of Linear Systems and Generalized Schr\"{o}dinger Bridges with Reference Refinement

TL;DR

This paper introduces a mutual information regularized density control method for linear systems, linking it to Schrödinger bridge problems and providing an optimization algorithm with Gaussian constraints.

Contribution

It proposes an alternating optimization algorithm for MI density control with Gaussian constraints and reveals its connection to generalized Schrödinger bridges.

Findings

Derived closed-form solutions for each step of the optimization algorithm.

Established the equivalence between MI density control and generalized Schrödinger bridge problems.

Abstract

We consider a mutual information (MI) regularized version of optimal density control of a discrete-time linear system. MI optimal control has been proposed as an extension of maximum entropy optimal control to trade off between control performance and benefits provided by stochastic inputs. MI regularization induces stochasticity in the policy, which poses challenges for applications of MI optimal control in safety-critical scenarios. To remedy this situation, we impose Gaussian density constraints at specified times to directly control state uncertainty. For this MI optimal density control problem, we propose an alternating optimization algorithm and derive the closed form of each step in the algorithm. In addition, we reveal that the alternating optimization of the MI optimal density control problem coincides with that of the so-called generalized Schr\"{o}dinger bridge problem associated with the discrete-time linear system.