Higher fidelity perceptual image and video compression with a latent conditioned residual denoising diffusion model

TL;DR

This paper introduces a hybrid perceptual image and video compression method that combines a decoder network with a latent conditioned diffusion model to improve fidelity while maintaining high perceptual quality, outperforming previous diffusion-based methods.

Contribution

It extends diffusion-based perceptual compression with a decoder to reduce distortion, achieving significant PSNR improvements without sacrificing perceptual quality.

Findings

Up to +2dB PSNR improvement on benchmarks

Maintains comparable LPIPS and FID scores

Easily extendable to video compression

Abstract

Denoising diffusion models achieved impressive results on several image generation tasks often outperforming GAN based models. Recently, the generative capabilities of diffusion models have been employed for perceptual image compression, such as in CDC. A major drawback of these diffusion-based methods is that, while producing impressive perceptual quality images they are dropping in fidelity/increasing the distortion to the original uncompressed images when compared with other traditional or learned image compression schemes aiming for fidelity. In this paper, we propose a hybrid compression scheme optimized for perceptual quality, extending the approach of the CDC model with a decoder network in order to reduce the impact on distortion metrics such as PSNR. After using the decoder network to generate an initial image, optimized for distortion, the latent conditioned diffusion model refines the reconstruction for perceptual quality by predicting the residual. On standard benchmarks, we achieve up to +2dB PSNR fidelity improvements while maintaining comparable LPIPS and FID perceptual scores when compared with CDC. Additionally, the approach is easily extensible to video compression, where we achieve similar results.