Feature-Preserving Rate-Distortion Optimization in Image Coding for Machines

TL;DR

This paper introduces a novel rate-distortion optimization method for image coding that preserves features relevant to machine analysis, utilizing Jacobian approximations for improved compression efficiency and downstream task performance.

Contribution

It proposes a feature-preserving RDO approach based on Jacobian approximation and localization, addressing neural network non-linearities and global information requirements.

Findings

Achieves bit-rate savings compared to traditional methods.

Maintains downstream task accuracy with less complexity.

Operates effectively in the transform domain.

Abstract

With the increasing number of images and videos consumed by computer vision algorithms, compression methods are evolving to consider both perceptual quality and performance in downstream tasks. Traditional codecs can tackle this problem by performing rate-distortion optimization (RDO) to minimize the distance at the output of a feature extractor. However, neural network non-linearities can make the rate-distortion landscape irregular, leading to reconstructions with poor visual quality even for high bit rates. Moreover, RDO decisions are made block-wise, while the feature extractor requires the whole image to exploit global information. In this paper, we address these limitations in three steps. First, we apply Taylor's expansion to the feature extractor, recasting the metric as an input-dependent squared error involving the Jacobian matrix of the neural network. Second, we make a localization assumption to compute the metric block-wise. Finally, we use randomized dimensionality reduction techniques to approximate the Jacobian. The resulting expression is monotonic with the rate and can be evaluated in the transform domain. Simulations with AVC show that our approach provides bit-rate savings while preserving accuracy in downstream tasks with less complexity than using the feature distance directly.