Analytical and Numerical Characterizations of Shannon Ordering for Discrete Memoryless Channels

TL;DR

This paper explores Shannon's partial ordering of discrete memoryless channels, providing conditions for channel inclusion, equivalence, and an efficient algorithm for determining inclusion using convex optimization and sparsity techniques.

Contribution

It introduces new majorization-based conditions for channel inclusion, establishes equivalence with permutation-based channel equivalence, and develops an iterative algorithm leveraging sparsity.

Findings

Channel inclusion can be characterized by majorization conditions.

Channel equivalence under Shannon ordering corresponds to permutation of symbols.

An iterative algorithm effectively determines channel inclusion using sparsity.

Abstract

This paper studies several problems concerning channel inclusion, which is a partial ordering between discrete memoryless channels (DMCs) proposed by Shannon. Specifically, majorization-based conditions are derived for channel inclusion between certain DMCs. Furthermore, under general conditions, channel equivalence defined through Shannon ordering is shown to be the same as permutation of input and output symbols. The determination of channel inclusion is considered as a convex optimization problem, and the sparsity of the weights related to the representation of the worse DMC in terms of the better one is revealed when channel inclusion holds between two DMCs. For the exploitation of this sparsity, an effective iterative algorithm is established based on modifying the orthogonal matching pursuit algorithm.