Information Nonanticipative Rate Distortion Function and Its Applications

TL;DR

This paper explores the nonanticipative Rate Distortion Function's applications in zero-delay coding, bounds on performance, and rate loss analysis, using Gaussian and binary sources to demonstrate theoretical and practical implications.

Contribution

It derives solutions for the nonanticipative RDF for Gaussian and binary sources, linking it to noncausal RDF and analyzing rate loss in zero-delay and causal codes.

Findings

Derived nonanticipative RDF solutions for Gaussian and binary sources.

Quantified rate loss of causal and zero-delay codes compared to noncausal codes.

Established equivalence between nonanticipative RDF and nonanticipatory epsilon-entropy.

Abstract

This paper investigates applications of nonanticipative Rate Distortion Function (RDF) in a) zero-delay Joint Source-Channel Coding (JSCC) design based on average and excess distortion probability, b) in bounding the Optimal Performance Theoretically Attainable (OPTA) by noncausal and causal codes, and computing the Rate Loss (RL) of zero-delay and causal codes with respect to noncausal codes. These applications are described using two running examples, the Binary Symmetric Markov Source with parameter p, (BSMS(p)) and the multidimensional partially observed Gaussian-Markov source. For the multidimensional Gaussian-Markov source with square error distortion, the solution of the nonanticipative RDF is derived, its operational meaning using JSCC design via a noisy coding theorem is shown by providing the optimal encoding-decoding scheme over a vector Gaussian channel, and the RL of causal and zero-delay codes with respect to noncausal codes is computed. For the BSMS(p) with Hamming distortion, the solution of the nonanticipative RDF is derived, the RL of causal codes with respect to noncausal codes is computed, and an uncoded noisy coding theorem based on excess distortion probability is shown. The information nonanticipative RDF is shown to be equivalent to the nonanticipatory epsilon-entropy, which corresponds to the classical RDF with an additional causality or nonanticipative condition imposed on the optimal reproduction conditional distribution.