On Self-Adaptive Perception Loss Function for Sequential Lossy Compression

TL;DR

This paper introduces a self-adaptive perception loss function for sequential lossy compression that improves realism and temporal correlation handling, supported by theoretical analysis and experiments on video datasets.

Contribution

It proposes a novel perception loss function considering joint source and reconstruction distribution, with theoretical analysis and practical validation.

Findings

The new PLF avoids error-permanence phenomenon.

It better exploits temporal correlations in reconstructions.

Experimental results show improved perceptual quality.

Abstract

We consider causal, low-latency, sequential lossy compression, with mean squared-error (MSE) as the distortion loss, and a perception loss function (PLF) to enhance the realism of reconstructions. As the main contribution, we propose and analyze a new PLF that considers the joint distribution between the current source frame and the previous reconstructions. We establish the theoretical rate-distortion-perception function for first-order Markov sources and analyze the Gaussian model in detail. From a qualitative perspective, the proposed metric can simultaneously avoid the error-permanence phenomenon and also better exploit the temporal correlation between high-quality reconstructions. The proposed metric is referred to as self-adaptive perception loss function (PLF-SA), as its behavior adapts to the quality of reconstructed frames. We provide a detailed comparison of the proposed perception loss function with previous approaches through both information theoretic analysis as well as experiments involving moving MNIST and UVG datasets.