The Infinite-message Limit of Two-terminal Interactive Source Coding

TL;DR

This paper characterizes the limit of the sum-rate-distortion function as the number of messages approaches infinity in a two-terminal source coding problem, introducing a convex-geometric approach that enables new bounds and insights.

Contribution

It provides a novel convex-geometric characterization of the infinite-message sum-rate-distortion function, enabling analysis without finite-message limits and solving longstanding open questions.

Findings

Closed-form characterization for Boolean AND function.

Infinite messages can strictly outperform single-message Wyner-Ziv rate.

New iterative algorithm for finite-message sum-rate-distortion evaluation.

Abstract

A two-terminal interactive function computation problem with alternating messages is studied within the framework of distributed block source coding theory. For any finite number of messages, a single-letter characterization of the sum-rate-distortion function was established in previous works using standard information-theoretic techniques. This, however, does not provide a satisfactory characterization of the infinite-message limit, which is a new, unexplored dimension for asymptotic-analysis in distributed block source coding involving potentially an infinite number of infinitesimal-rate messages. In this paper, the infinite-message sum-rate-distortion function, viewed as a functional of the joint source pmf and the distortion levels, is characterized as the least element of a partially ordered family of functionals having certain convex-geometric properties. The new characterization does not involve evaluating the infinite-message limit of a finite-message sum-rate-distortion expression. This characterization leads to a family of lower bounds for the infinite-message sum-rate-distortion expression and a simple criterion to test the optimality of any achievable infinite-message sum-rate-distortion expression. For computing the amplewise Boolean AND function, the infinite-message minimum sum-rates are characterized in closed analytic form. These sum-rates are shown to be achievable using infinitely many infinitesimal-rate messages. The new convex-geometric characterization is used to develop an iterative algorithm for evaluating any finite-message sumrate-distortion function. It is also used to construct the first examples which demonstrate that for lossy source reproduction, two messages can strictly improve the one-message Wyner-Ziv rate-distortion function settling an unresolved question from a 1985 paper.