Exploring Compressed Image Representation as a Perceptual Proxy: A Study

TL;DR

This paper demonstrates that compressed image representations can effectively serve as perceptual proxies, enabling high-quality image compression and perceptual quality assessment without additional neural networks.

Contribution

It introduces a joint training approach for image compression and classification, showing compressed representations can predict perceptual distances and serve as perceptual loss functions.

Findings

Compressed latent representations predict perceptual distances accurately.

Off-the-shelf neural encoders can be used as perceptual loss networks.

The approach reduces the need for specialized perceptual models.

Abstract

We propose an end-to-end learned image compression codec wherein the analysis transform is jointly trained with an object classification task. This study affirms that the compressed latent representation can predict human perceptual distance judgments with an accuracy comparable to a custom-tailored DNN-based quality metric. We further investigate various neural encoders and demonstrate the effectiveness of employing the analysis transform as a perceptual loss network for image tasks beyond quality judgments. Our experiments show that the off-the-shelf neural encoder proves proficient in perceptual modeling without needing an additional VGG network. We expect this research to serve as a valuable reference developing of a semantic-aware and coding-efficient neural encoder.