Maximum Entropy Density Control of Discrete-Time Linear Systems with Quadratic Cost

TL;DR

This paper develops a method for controlling the distribution of states in discrete-time linear systems to a target Gaussian distribution by solving coupled Riccati equations, providing explicit formulas for optimal policies.

Contribution

It introduces a novel maximum entropy density control framework for linear systems with quadratic costs, deriving closed-form solutions and analyzing the unregularized limit.

Findings

Closed-form optimal policies derived from Riccati equations

Dual forward and backward system solutions for density control

Explicit unregularized density control formula in the limit

Abstract

This paper addresses the problem of steering the distribution of the state of a discrete-time linear system to a given target distribution while minimizing an entropy-regularized cost functional. This problem is called a maximum entropy density control problem. Specifically, the running cost is given by quadratic forms of the state and the control input, and the initial and target distributions are Gaussian. We first reveal that our problem boils down to solving two Riccati difference equations coupled through their boundary values. Based on them, we give the closed-form expression of the unique optimal policy. Next, we show that the optimal density control of a backward system can be obtained simultaneously with the forward-time optimal policy. The backward solution gives another expression of the forward solution. Finally, by considering the limit where the entropy regularization vanishes, we derive the unregularized density control in closed form.