Computability of the Optimizer for Rate Distortion Functions

TL;DR

This paper investigates the computability of the optimizer in rate distortion functions, revealing that while the rate distortion function itself is usually computable, the optimizer often is not, even for simple measures.

Contribution

It demonstrates that the optimizer for rate distortion functions can be non-computable, extending known results about other information theoretic optimizers.

Findings

Rate distortion functions are generally computable.

The optimizer for rate distortion functions can be non-computable.

Non-computability persists even with simple distortion measures.

Abstract

Rate distortion theory treats the problem of encoding a source with minimum codebook size while at the same time allowing for a certain amount of errors in the reconstruction measured by a fidelity criterion and distortion level. Similar to the channel coding problem the optimal rate of the codebook with respect to the blocklength is given by a convex optimization problem involving information theoretic quantities like mutual information. The value of the rate in dependence of the distortion level as well as the optimizer used in the codebook construction are of theoretical and practical importance in communication and information theory. In this paper the behavior of the rate distortion function regarding the computability of the optimizing test channel is investigated. We find that comparable with known results about the optimizer for other information theoretic problems a similar result is found to be true also regarding the computability of the optimizer for rate distortion functions. It turns out that while the rate distortion function is usually computable the optimizer for this problem is in general non-computable even for simple distortion measures.