Feedback Capacity of MIMO Gaussian Channels

TL;DR

This paper derives a computable, finite-dimensional convex optimization expression for the feedback capacity of MIMO Gaussian channels with colored noise generated by a state-space model, unifying information theory and control tools.

Contribution

It provides the first explicit, computable formula for feedback capacity in MIMO Gaussian channels with colored noise, using a novel sequential convex optimization approach.

Findings

Feedback capacity expressed as a finite-dimensional convex optimization problem.

Time-invariant policies are optimal even for non-stationary noise.

Constructs a simple, capacity-achieving coding scheme for scalar channels.

Abstract

Finding a computable expression for the feedback capacity of channels with colored Gaussian, additive noise is a long standing open problem. In this paper, we solve this problem in the scenario where the channel has multiple inputs and multiple outputs (MIMO) and the noise process is generated as the output of a time-invariant state-space model. Our main result is a computable expression for the feedback capacity in terms of a finite-dimensional convex optimization. The solution to the feedback capacity problem is obtained by formulating the finite-block counterpart of the capacity problem as a \emph{sequential convex optimization problem} which leads in turn to a single-letter upper bound. This converse derivation integrates tools and ideas from information theory, control, filtering and convex optimization. A tight lower bound is realized by optimizing over a family of time-invariant policies thus showing that time-invariant inputs are optimal even when the noise process may not be stationary. The optimal time-invariant policy is used to construct a capacity-achieving and simple coding scheme for scalar channels, and its analysis reveals an interesting relation between a smoothing problem and the feedback capacity expression.