Algorithm Families for Computing Information-Theoretic Forms of Strong Converse Exponents in Channel Coding and Lossy Source Coding

TL;DR

This paper explores algorithms for computing strong converse exponents in channel and source coding, establishing their equivalence without KKT condition evaluation and introducing new algorithm families for lossy source coding.

Contribution

It clarifies the relation between different representations of exponents and introduces new algorithm families that unify existing methods and simplify computations.

Findings

Proves the equivalence of Arimoto's and Dueck-Korner's exponents without KKT conditions.

Introduces a new algorithm family for lossy source coding exponents.

Shows convergence of algorithms implies the matching of different exponent expressions.

Abstract

The error exponent of a discrete memoryless channel is expressed in two forms. One is Gallager's expression with a positive slope parameter and the other is Csiszar and Korner's information-theoretic representation expressed using the mutual information and the relative entropy. They differ in appearance, and existing methods to prove their agreement are not elementary, as they require an evaluation of the KKT conditions that the optimal distribution must satisfy. Similarly, there are two types of expressions for the strong converse exponent. They are Arimoto's expression with a negative slope parameter and Dueck and Korner's information-theoretic expression. The purpose of this paper is to clarify the relation between two ways of representing exponents, i.e., representations using slope parameters and those using information-theoretic quantities, from the viewpoint of algorithms for computing exponents. Arimoto's algorithm is based on expression using slope parameters, while the authors' and Tridenski and Zamir's algorithms are based on Dueck and Korner's information-theoretic expression. An algorithm family that includes the above two algorithms as special cases was recently proposed. This paper clarifies that the convergence of Tridenski and Zamir's algorithm proves the match of Arimoto's and Dueck and Korner's exponents. We discuss another family of algorithms and, using the surrogate objective function used therein, prove that the two expressions of the error exponent coincide. Evaluation of the KKT condition is not needed in this proof. We then discuss the computation of the error and correct decoding probability exponents in lossy source coding. A new algorithm family for computing the source coding strong converse exponent is defined. The convergence of a member of the algorithm family implies the match of the two expressions of the strong converse exponent.