The Rate-Distortion Risk in Estimation from Compressed Data

TL;DR

This paper investigates the theoretical relationship between the risk of estimating a latent signal from lossy compressed data and the rate-distortion function, providing conditions for their asymptotic equivalence and guiding estimator design.

Contribution

It establishes conditions under which estimation risk from compressed data matches the rate-distortion risk, enabling estimator design without knowing the compression details.

Findings

Asymptotic equivalence between operational risk and RD risk under certain conditions.

Explicit RD risk expressions for discrete and Gaussian data.

Comparison of estimator performance to full knowledge optimal coding.

Abstract

Consider the problem of estimating a latent signal from a lossy compressed version of the data when the compressor is agnostic to the relation between the signal and the data. This situation arises in a host of modern applications when data is transmitted or stored prior to determining the downstream inference task. Given a bitrate constraint and a distortion measure between the data and its compressed version, let us consider the joint distribution achieving Shannon's rate-distortion (RD) function. Given an estimator and a loss function associated with the downstream inference task, define the rate-distortion risk as the expected loss under the RD-achieving distribution. We provide general conditions under which the operational risk in estimating from the compressed data is asymptotically equivalent to the RD risk. The main theoretical tools to prove this equivalence are transportation-cost inequalities in conjunction with properties of compression codes achieving Shannon's RD function. Whenever such equivalence holds, a recipe for designing estimators from datasets undergoing lossy compression without specifying the actual compression technique emerges: design the estimator to minimize the RD risk. Our conditions simplified in the special cases of discrete memoryless or multivariate normal data. For these scenarios, we derive explicit expressions for the RD risk of several estimators and compare them to the optimal source coding performance associated with full knowledge of the relation between the latent signal and the data.