Feedback Capacity of Stationary Gaussian Channels

TL;DR

This paper characterizes the feedback capacity of stationary Gaussian channels, proves the optimality of stationary coding schemes, and demonstrates that the Schalkwijk-Kailath scheme achieves capacity for certain noise spectra, solving a long-standing open problem.

Contribution

It provides a variational characterization of feedback capacity, proves stationarity of optimal schemes, and generalizes the Schalkwijk-Kailath scheme to ARMA noise spectra of any order.

Findings

Closed-form feedback capacity for ARMA(1) noise spectrum

Schalkwijk-Kailath scheme achieves capacity for ARMA(1)

Generalized scheme achieves capacity for ARMA(k) noise spectra

Abstract

The feedback capacity of additive stationary Gaussian noise channels is characterized as the solution to a variational problem. Toward this end, it is proved that the optimal feedback coding scheme is stationary. When specialized to the first-order autoregressive moving average noise spectrum, this variational characterization yields a closed-form expression for the feedback capacity. In particular, this result shows that the celebrated Schalkwijk-Kailath coding scheme achieves the feedback capacity for the first-order autoregressive moving average Gaussian channel, positively answering a long-standing open problem studied by Butman, Schalkwijk-Tiernan, Wolfowitz, Ozarow, Ordentlich, Yang-Kavcic-Tatikonda, and others. More generally, it is shown that a k-dimensional generalization of the Schalkwijk-Kailath coding scheme achieves the feedback capacity for any autoregressive moving average noise spectrum of order k. Simply put, the optimal transmitter iteratively refines the receiver's knowledge of the intended message.