A Simple Converse Proof and a Unified Capacity Formula for Channels with Input Constraints

TL;DR

This paper introduces a simple, unified capacity formula for channels with input constraints using a Lagrangian approach, applicable to both convex and non-convex problems, and extends the converse proof accordingly.

Contribution

It presents the Lagrangian Converse Proof method, providing a unified capacity formula that simplifies analysis for channels with input constraints, regardless of problem convexity.

Findings

The capacity formula is the minimum of a Lagrange dual function.

The method applies to various channels, including those with state information and feedback.

The approach extends to rate distortion theory.

Abstract

Given the single-letter capacity formula and the converse proof of a channel without constraints, we provide a simple approach to extend the results for the same channel but with constraints. The resulting capacity formula is the minimum of a Lagrange dual function. It gives an unified formula in the sense that it works regardless whether the problem is convex. If the problem is non-convex, we show that the capacity can be larger than the formula obtained by the naive approach of imposing constraints on the maximization in the capacity formula of the case without the constraints. The extension on the converse proof is simply by adding a term involving the Lagrange multiplier and the constraints. The rest of the proof does not need to be changed. We name the proof method the Lagrangian Converse Proof. In contrast, traditional approaches need to construct a better input distribution for convex problems or need to introduce a time sharing variable for non-convex problems. We illustrate the Lagrangian Converse Proof for three channels, the classic discrete time memoryless channel, the channel with non-causal channel-state information at the transmitter, the channel with limited channel-state feedback. The extension to the rate distortion theory is also provided.