State distributions and minimum relative entropy noise sequences in uncertain stochastic systems: the discrete time case

TL;DR

This paper develops a dissipativity theory for discrete-time stochastic systems under uncertain noise, introducing a robustness measure based on minimum relative entropy and solving related variational problems with applications to linear Gaussian systems.

Contribution

It formulates a new entropy-based robustness measure for stochastic systems and derives a Bellman equation for minimum entropy supply, including a closed-form solution for Gaussian linear systems.

Findings

Derived a Bellman equation for entropy-based robustness.

Established a Markov property for worst-case noise.

Provided a closed-form solution for Gaussian linear systems.

Abstract

The paper is concerned with a dissipativity theory and robust performance analysis of discrete-time stochastic systems driven by a statistically uncertain random noise. The uncertainty is quantified by the conditional relative entropy of the actual probability law of the noise with respect to a nominal product measure corresponding to a white noise sequence. We discuss a balance equation, dissipation inequality and superadditivity property for the corresponding conditional relative entropy supply as a function of time. The problem of minimizing the supply required to drive the system between given state distributions over a specified time horizon is considered. Such variational problems, involving entropy and probabilistic boundary conditions, are known in the literature as Schroedinger bridge problems. In application to control systems, this minimum required conditional relative entropy supply characterizes the robustness of the system with respect to an uncertain noise. We obtain a dynamic programming Bellman equation for the minimum required conditional relative entropy supply and establish a Markov property of the worst-case noise with respect to the state of the system. For multivariable linear systems with a Gaussian white noise sequence as the nominal noise model and Gaussian initial and terminal state distributions, the minimum required supply is obtained using an algebraic Riccati equation which admits a closed-form solution. We propose a computable robustness index for such systems in the framework of an entropy theoretic formulation of uncertainty and provide an example to illustrate this approach.