Rate-Distortion Bounds for an Epsilon-Insensitive Distortion Measure

TL;DR

This paper analyzes the rate-distortion function for an epsilon-insensitive loss, showing it exceeds the Shannon lower bound for certain sources and providing bounds that suggest the lower bound is a good approximation.

Contribution

It derives the rate-distortion function for epsilon-insensitive loss and establishes bounds for Laplacian and Gaussian sources, highlighting when the Shannon lower bound is tight.

Findings

Rate-distortion functions for Laplacian and Gaussian sources are strictly greater than Shannon bounds.

Analytically evaluable upper bounds for these sources are obtained.

Shannon lower bound approximates the rate-distortion function well in small distortion regimes.

Abstract

Direct evaluation of the rate-distortion function has rarely been achieved when it is strictly greater than its Shannon lower bound. In this paper, we consider the rate-distortion function for the distortion measure defined by an epsilon-insensitive loss function. We first present the Shannon lower bound applicable to any source distribution with finite differential entropy. Then, focusing on the Laplacian and Gaussian sources, we prove that the rate-distortion functions of these sources are strictly greater than their Shannon lower bounds and obtain analytically evaluable upper bounds for the rate-distortion functions. Small distortion limit and numerical evaluation of the bounds suggest that the Shannon lower bound provides a good approximation to the rate-distortion function for the epsilon-insensitive distortion measure.