Covariance Steering of Discrete-Time Stochastic Linear Systems Based on Distribution Distance Terminal Costs

TL;DR

This paper develops a method for controlling discrete-time stochastic linear systems to steer their terminal state distributions close to a desired Gaussian, using distribution distance measures like Wasserstein and KL divergence, solved via convex optimization techniques.

Contribution

It introduces a novel approach to stochastic control using distribution distance-based terminal costs and reformulates the problem as a difference-of-convex functions optimization.

Findings

Effective control policies for distribution steering demonstrated.

Comparison of Wasserstein and KL divergence approaches.

Numerical simulations show computational efficiency.

Abstract

We consider a class of stochastic optimal control problems for discrete-time stochastic linear systems which seek for control policies that will steer the probability distribution of the terminal state of the system close to a desired Gaussian distribution. In our problem formulation, the closeness between the terminal state distribution and the desired (goal) distribution is measured in terms of the squared Wasserstein distance which is associated with a corresponding terminal cost term. We recast the stochastic optimal control problem as a finite-dimensional nonlinear program and we show that its performance index can be expressed as the difference of two convex functions. This representation of the performance index allows us to find local minimizers of the original nonlinear program via the so-called convex-concave procedure [1]. Subsequently, we consider a similar problem but this time we use a terminal cost that corresponds to the KL divergence. Finally, we present non-trivial numerical simulations to demonstrate the proposed techniques and compare them in terms of computation time.