A Deterministic Annealing Approach to Optimization of Zero-delay Source-Channel Codes

TL;DR

This paper introduces a deterministic annealing algorithm for globally optimizing zero-delay source-channel codes with side information, outperforming traditional greedy and noisy channel relaxation methods in complex scenarios.

Contribution

The paper develops a novel deterministic annealing approach for optimizing source-channel codes with side information, overcoming local minima issues of previous methods.

Findings

Deterministic annealing outperforms greedy optimization.

Proposed method achieves better global optimality.

Numerical results demonstrate superior performance in complex settings.

Abstract

This paper studies optimization of zero-delay source-channel codes, and specifically the problem of obtaining globally optimal transformations that map between the source space and the channel space, under a given transmission power constraint and for the mean square error distortion. Particularly, we focus on the setting where the decoder has access to side information, whose cost surface is known to be riddled with local minima. Prior work derived the necessary conditions for optimality of the encoder and decoder mappings, along with a greedy optimization algorithm that imposes these conditions iteratively, in conjunction with the heuristic "noisy channel relaxation" method to mitigate poor local minima. While noisy channel relaxation is arguably effective in simple settings, it fails to provide accurate global optimization results in more complicated settings including the decoder with side information as considered in this paper. We propose a global optimization algorithm based on the ideas of "deterministic annealing"- a non-convex optimization method, derived from information theoretic principles with analogies to statistical physics, and successfully employed in several problems including clustering, vector quantization and regression. We present comparative numerical results that show strict superiority of the proposed algorithm over greedy optimization methods as well as over the noisy channel relaxation.