Root Tracking for Rate-Distortion: Approximating a Solution Curve with Higher Implicit Multivariate Derivatives

TL;DR

This paper introduces algorithms that leverage the dynamics of the rate-distortion curve, using higher-order derivatives and bifurcation analysis, to accurately track optimal solutions in lossy data compression.

Contribution

It develops a novel method to approximate the entire rate-distortion solution curve by exploiting implicit derivatives and bifurcation analysis, advancing understanding of solution dynamics.

Findings

Algorithms successfully track the rate-distortion curve.

Implicit derivatives enable approximation of neighboring solutions.

Bifurcation analysis guarantees solution optimality and detects failures.

Abstract

The rate-distortion curve captures the fundamental tradeoff between compression length and resolution in lossy data compression. However, it conceals the underlying dynamics of optimal source encodings or test channels. We argue that these typically follow a piecewise smooth trajectory as the source information is compressed. These smooth dynamics are interrupted at bifurcations, where solutions change qualitatively. Sub-optimal test channels may collide or exchange optimality there, for example. There is typically a plethora of sub-optimal solutions, which stems from restrictions of the reproduction alphabet. We devise a family of algorithms that exploits the underlying dynamics to track a given test channel along the rate-distortion curve. To that end, we express implicit derivatives at the roots of a non-linear operator by higher derivative tensors. Providing closed-form formulae for the derivative tensors of Blahut's algorithm thus yields implicit derivatives of arbitrary order at a given test channel, thereby approximating others in its vicinity. Finally, our understanding of bifurcations guarantees the optimality of the root being traced, under mild assumptions, while allowing us to detect when our assumptions fail. Beyond the interest in rate distortion, this is an example of how understanding a problem's bifurcations can be translated to a numerical algorithm.