On Perceptual Lossy Compression: The Cost of Perceptual Reconstruction and An Optimal Training Framework

TL;DR

This paper reveals the theoretical cost of perfect perceptual quality in lossy compression doubles the minimal MSE distortion and introduces a new training framework using GANs conditioned on an MSE-optimized encoder to achieve optimal perceptual compression.

Contribution

It provides a theoretical analysis of the cost of perceptual quality and proposes a novel GAN-based training framework for optimal perceptual lossy compression.

Findings

Achieving perfect perception doubles the minimal MSE distortion.

An optimal encoder for rate-distortion is also optimal for perceptual compression.

The proposed GAN-based framework outperforms traditional methods.

Abstract

Lossy compression algorithms are typically designed to achieve the lowest possible distortion at a given bit rate. However, recent studies show that pursuing high perceptual quality would lead to increase of the lowest achievable distortion (e.g., MSE). This paper provides nontrivial results theoretically revealing that, \textit{1}) the cost of achieving perfect perception quality is exactly a doubling of the lowest achievable MSE distortion, \textit{2}) an optimal encoder for the "classic" rate-distortion problem is also optimal for the perceptual compression problem, \textit{3}) distortion loss is unnecessary for training a perceptual decoder. Further, we propose a novel training framework to achieve the lowest MSE distortion under perfect perception constraint at a given bit rate. This framework uses a GAN with discriminator conditioned on an MSE-optimized encoder, which is superior over the traditional framework using distortion plus adversarial loss. Experiments are provided to verify the theoretical finding and demonstrate the superiority of the proposed training framework.