Zero-Delay Lossy Coding of Linear Vector Markov Sources: Optimality of Stationary Codes and Near Optimality of Finite Memory Codes

TL;DR

This paper establishes the existence and structure of optimal stationary zero-delay codes for linear Markov sources under quadratic distortion, and quantifies how finite-memory codes approach infinite-horizon optimality.

Contribution

It proves the existence of deterministic stationary coding policies for infinite horizon problems and provides explicit convergence rates for finite-memory codes.

Findings

Optimal stationary codes exist for the infinite horizon problem.

Finite-memory codes approach infinite-horizon optimality at a rate of O(1/T).

Finite window codes are nearly optimal among all zero-delay codes.

Abstract

Optimal zero-delay coding (quantization) of $\mathbb{R}^d$-valued linearly generated Markov sources is studied under quadratic distortion. The structure and existence of deterministic and stationary coding policies that are optimal for the infinite horizon average cost (distortion) problem are established. Prior results studying the optimality of zero-delay codes for Markov sources for infinite horizons either considered finite alphabet sources or, for the $\mathbb{R}^d$-valued case, only showed the existence of deterministic and non-stationary Markov coding policies or those which are randomized. In addition to existence results, for finite blocklength (horizon) $T$ the performance of an optimal coding policy is shown to approach the infinite time horizon optimum at a rate $O(\frac{1}{T})$. This gives an explicit rate of convergence that quantifies the near-optimality of finite window (finite-memory) codes among all optimal zero-delay codes.