Entropy-Constrained Maximizing Mutual Information Quantization

TL;DR

This paper develops a polynomial-time algorithm for optimal quantization of binary input channels to maximize mutual information under entropy constraints, applicable to both discrete and continuous outputs, with theoretical and numerical validation.

Contribution

Introduces a polynomial-time algorithm for globally optimal quantization maximizing mutual information with entropy constraints for binary channels.

Findings

Algorithm finds the global optimal quantizer efficiently.

Optimality conditions for single threshold quantizers in continuous output channels.

Numerical results validate the theoretical approach.

Abstract

In this paper, we investigate the quantization of the output of a binary input discrete memoryless channel that maximizing the mutual information between the input and the quantized output under an entropy-constrained of the quantized output. A polynomial time algorithm is introduced that can find the truly global optimal quantizer. These results hold for binary input channels with an arbitrary number of quantized output. Finally, we extend these results to binary input continuous output channels and show a sufficient condition such that a single threshold quantizer is an optimal quantizer. Both theoretical results and numerical results are provided to justify our techniques.