PICD: Versatile Perceptual Image Compression with Diffusion Rendering

TL;DR

PICD introduces a versatile perceptual image compression method that uses diffusion rendering to effectively compress both screen and natural images, especially improving text preservation in screen content.

Contribution

The paper presents a novel compression framework that encodes text and images separately and employs diffusion models with multi-level conditioning for high-quality perceptual reconstruction.

Findings

Outperforms existing perceptual codecs in text accuracy and visual quality

Effective as a perceptual codec for natural images without text conditions

Utilizes a multi-level diffusion model conditioning strategy

Abstract

Recently, perceptual image compression has achieved significant advancements, delivering high visual quality at low bitrates for natural images. However, for screen content, existing methods often produce noticeable artifacts when compressing text. To tackle this challenge, we propose versatile perceptual screen image compression with diffusion rendering (PICD), a codec that works well for both screen and natural images. More specifically, we propose a compression framework that encodes the text and image separately, and renders them into one image using diffusion model. For this diffusion rendering, we integrate conditional information into diffusion models at three distinct levels: 1). Domain level: We fine-tune the base diffusion model using text content prompts with screen content. 2). Adaptor level: We develop an efficient adaptor to control the diffusion model using compressed image and text as input. 3). Instance level: We apply instance-wise guidance to further enhance the decoding process. Empirically, our PICD surpasses existing perceptual codecs in terms of both text accuracy and perceptual quality. Additionally, without text conditions, our approach serves effectively as a perceptual codec for natural images.